Add E2E smoke test by mschoe · Pull Request #74 · camunda/c8ctl

mschoe · 2026-02-19T10:24:27Z

end-to-end smoke test as a shell script that validates core CLI functionality

Copilot

Pull request overview

Adds a lightweight shell-based end-to-end smoke test intended to quickly validate core c8ctl CLI workflows against a local Camunda 8 instance.

Changes:

Add tests/e2e/smoke-test.sh to exercise common CLI paths (help, deploy/run/create/list, user tasks, message correlation, search, profiles).
Add new BPMN fixtures used by the smoke test (user task, timer event, message correlation).
Document running the smoke test in tests/e2e/README.md.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/e2e/smoke-test.sh	New bash smoke test driving the CLI through key scenarios
tests/e2e/README.md	Usage documentation for the smoke test script
tests/fixtures/simple-user-task.bpmn	New BPMN fixture for user-task scenario
tests/fixtures/simple-timer-event.bpmn	New BPMN fixture for timer-event scenario
tests/fixtures/simple-message-correlation.bpmn	New BPMN fixture for message correlation scenario

tests/e2e/smoke-test.sh

Copilot · 2026-02-19T10:28:56Z

tests/e2e/smoke-test.sh

+cleanup_session() {
+    # Clean session state between tests
+    local config_dir="${XDG_CONFIG_HOME:-$HOME/.config}/c8ctl"
+    if [ -f "$config_dir/session.json" ]; then
+        rm -f "$config_dir/session.json"
+    fi
+}


The smoke test deletes the user's real ~/.config/c8ctl/session.json to reset state. Running this locally can disrupt a developer's configured environment. Consider isolating the test by setting XDG_CONFIG_HOME (and/or HOME) to a temporary directory for the entire script and cleaning it up on exit, instead of deleting files from the user's actual config directory.

@copilot open a new pull request to apply changes based on this feedback

Copilot · 2026-02-19T10:28:56Z

tests/e2e/smoke-test.sh

+# Test 9: Profile management
+test_start "profile management"
+cleanup_session
+# Step 1: Create the profile
+if ADD_OUTPUT=$(c8_cmd add profile prod --baseUrl=https://camunda.example.com --clientId=xxx --clientSecret=yyy 2>&1); then
+    debug_output "$ADD_OUTPUT"
+    if ! echo "$ADD_OUTPUT" | grep -q "Profile 'prod' added"; then
+        test_fail "expected success message for profile addition"
+    else
+        # Step 2: Validate the existence of the profile
+        if LIST_OUTPUT=$(c8_cmd 0 list profiles 2>&1); then
+            debug_output "$LIST_OUTPUT"
+            if ! echo "$LIST_OUTPUT" | grep -q "prod"; then
+                test_fail "profile 'prod' not found in list after creation"
+            elif ! echo "$LIST_OUTPUT" | grep -q "https://camunda.example.com"; then
+                test_fail "base URL not found in profile list"
+            else
+                # Step 3: Delete the profile
+                if REMOVE_OUTPUT=$(c8_cmd remove profile prod 2>&1); then
+                    debug_output "$REMOVE_OUTPUT"
+                    if ! echo "$REMOVE_OUTPUT" | grep -q "Profile 'prod' removed"; then
+                        test_fail "expected success message for profile removal"
+                    else
+                        # Step 4: Evaluate the deleted list
+                        if LIST_AFTER=$(c8_cmd 0 list profiles 2>&1); then
+                            debug_output "$LIST_AFTER"
+                            if echo "$LIST_AFTER" | grep -q "prod"; then
+                                test_fail "profile 'prod' still appears in list after deletion"
+                            else


This test uses a hard-coded profile name prod. If a developer already has a prod profile, the test may fail (or overwrite/alter expectations) and it will still mutate their config if the script exits early. Use a unique, script-scoped profile name (e.g., with a random/timestamp suffix) and/or run with an isolated config directory to avoid collisions and side effects.

@copilot change the profile name to e2e-smoke-test

tests/e2e/smoke-test.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot · 2026-02-20T09:00:34Z

@mschoe I've opened a new pull request, #76, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot · 2026-02-20T09:01:08Z

@mschoe I've opened a new pull request, #77, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot · 2026-02-20T09:02:21Z

@mschoe I've opened a new pull request, #78, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

Copilot · 2026-02-20T09:05:24Z

tests/e2e/smoke-test.sh

+# CLI command
+C8="node src/index.ts"
+


The script assumes it is executed from the repository root (relative paths like src/index.ts and tests/fixtures/...). If run from another working directory, it will fail. Consider resolving the repo root from the script location (e.g., using BASH_SOURCE[0]) and either cd there or build absolute paths for C8 and fixtures.

Copilot · 2026-02-20T09:05:25Z

tests/e2e/smoke-test.sh

+test_pass() {
+    TESTS_PASSED=$((TESTS_PASSED + 1))
+    echo -e "${GREEN}✓ PASS${NC}"
+}
+


test_pass is defined twice; the second definition overwrites the first and makes the earlier block dead code. Remove the duplicate or consolidate to a single definition to avoid confusion when editing the script.

Suggested change

test_pass() {

TESTS_PASSED=$((TESTS_PASSED + 1))

echo -e "${GREEN}✓ PASS${NC}"

}

Copilot · 2026-02-20T09:05:25Z

tests/e2e/smoke-test.sh

+    debug_output "$OUTPUT"
+    if echo "$OUTPUT" | grep -q "Deployment successful"; then
+        # Extract process definition key from the output table (last column before the end)
+        PROCESS_KEY=$(echo "$OUTPUT" | grep "simple-process" | awk '{print $NF}')


With set -euo pipefail, this grep | awk pipeline will terminate the whole script if grep finds no match (exit code 1), preventing the test from recording a controlled failure. Guard the pipeline (e.g., ... || true) or restructure extraction so a missing match results in an empty variable and a test_fail, not an immediate exit.

Suggested change

PROCESS_KEY=$(echo "$OUTPUT" | grep "simple-process" | awk '{print $NF}')

PROCESS_KEY=$(echo "$OUTPUT" | awk '/simple-process/ {print $NF}')

Copilot · 2026-02-20T09:05:25Z

tests/e2e/smoke-test.sh

+        INSTANCE_KEY=$(echo "$RUN_OUTPUT" | grep "Process instance created" | grep -o '\[Key: [0-9]*\]' | grep -o '[0-9]*')
+


This key-extraction pipeline can cause the entire script to exit under set -euo pipefail if any grep returns no match (exit code 1). It should be written so extraction failure yields an empty INSTANCE_KEY and the test can call test_fail, rather than aborting the smoke test early.

Suggested change

INSTANCE_KEY=$(echo "$RUN_OUTPUT" | grep "Process instance created" | grep -o '\[Key: [0-9]*\]' | grep -o '[0-9]*')

INSTANCE_KEY=""

if [[ "$RUN_OUTPUT" =~ \[Key:\ ([0-9]+)\] ]]; then

INSTANCE_KEY="${BASH_REMATCH[1]}"

fi

Copilot · 2026-02-20T09:05:25Z

tests/e2e/smoke-test.sh

+        # List user tasks and extract the key (wait time handled by c8_cmd)
+        if LIST_UT=$(c8_cmd 0 list ut 2>&1); then
+            debug_output "$LIST_UT"
+            UT_KEY=$(echo "$LIST_UT" | grep -v "^Key\|^---\|^No user" | awk 'NF {print $1}' | head -n1)


grep -v returns exit code 1 when it outputs no lines; with set -euo pipefail that will abort the script if there are currently no user tasks (which is exactly the condition you want to detect). Make this extraction tolerant of empty output (e.g., allow grep to fail) and then handle the empty UT_KEY case with test_fail as you already do.

Suggested change

UT_KEY=$(echo "$LIST_UT" | grep -v "^Key\|^---\|^No user" | awk 'NF {print $1}' | head -n1)

UT_KEY=$(

echo "$LIST_UT" \

| { grep -v "^Key\|^---\|^No user" || true; } \

| awk 'NF {print $1}' \

| head -n1

)

Copilot · 2026-02-20T09:05:26Z

tests/e2e/README.md

+## Prerequisites
+
+- **Camunda 8** running at `http://localhost:8080`
+
+## Usage


The prerequisites mention Camunda 8 at http://localhost:8080, but the CLI’s default self-managed base URL is http://localhost:8080/v2 (and it falls back to demo/demo auth). Consider documenting the /v2 requirement and any expected auth defaults. Also, if the smoke test is intended to be safe to run locally, document (or enforce in the script) using C8CTL_DATA_DIR to isolate config so it doesn’t touch the user’s real ~/.config / ~/Library/... state.

Copilot · 2026-02-20T09:05:26Z

tests/e2e/smoke-test.sh

+cleanup_session() {
+    # Clean session state between tests
+    local config_dir="${XDG_CONFIG_HOME:-$HOME/.config}/c8ctl"
+    if [ -f "$config_dir/session.json" ]; then
+        rm -f "$config_dir/session.json"
+    fi
+}


cleanup_session hard-codes the Linux config path ($XDG_CONFIG_HOME/~/.config) and will not work on macOS (where c8ctl stores config under ~/Library/Application Support/c8ctl). It also deletes a real user session file, which is a risky side effect for a smoke test. Prefer isolating test state by setting C8CTL_DATA_DIR (or XDG_CONFIG_HOME) to a temporary directory at the start of the script and cleaning it up via a trap on exit, instead of deleting from the user’s config directory.

test: Add e2e tests

e74f2a3

Copilot AI review requested due to automatic review settings February 19, 2026 10:24

Copilot started reviewing on behalf of mschoe February 19, 2026 10:24 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Update tests/e2e/smoke-test.sh

8292b99

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 20, 2026 08:59

Update tests/e2e/smoke-test.sh

0e72094

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot started reviewing on behalf of mschoe February 20, 2026 09:00 View session

Copilot AI mentioned this pull request Feb 20, 2026

refactor(e2e): replace fixed sleep with polling helpers in smoke test #76

Draft

Copilot AI mentioned this pull request Feb 20, 2026

fix: isolate smoke test from user's real config directory #77

Draft

Copilot AI mentioned this pull request Feb 20, 2026

test(e2e): use unique profile name in smoke test to avoid collisions #78

Draft

Copilot AI reviewed Feb 20, 2026

View reviewed changes

	PROCESS_KEY=$(echo "$OUTPUT" \| grep "simple-process" \| awk '{print $NF}')
	PROCESS_KEY=$(echo "$OUTPUT" \| awk '/simple-process/ {print $NF}')

		INSTANCE_KEY=$(echo "$RUN_OUTPUT" \| grep "Process instance created" \| grep -o '\[Key: [0-9]\]' \| grep -o '[0-9]')

-        INSTANCE_KEY=$(echo "$RUN_OUTPUT" | grep "Process instance created" | grep -o '\[Key: [0-9]*\]' | grep -o '[0-9]*')
+        INSTANCE_KEY=""
+        if [[ "$RUN_OUTPUT" =~ \[Key:\ ([0-9]+)\] ]]; then
+            INSTANCE_KEY="${BASH_REMATCH[1]}"
+        fi

Comments

Conversation

mschoe commented Feb 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

mschoe Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

mschoe Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI commented Feb 20, 2026

Uh oh!

Copilot AI commented Feb 20, 2026

Uh oh!

Copilot AI commented Feb 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants