CLI help text shows wrong DEFAULT_STRENGTH (0.75 vs actual 1.0)

[Serhan-Asad]

Bug

The --strength CLI option help text says the default is 0.75, but the actual default in pdd/__init__.py is 1.0.

Evidence

• pdd/__init__.py:12: DEFAULT_STRENGTH = 1.0
• pdd/core/cli.py:220: Help text says "Default: 0.75 or .pddrc value"

Impact

Users who don't explicitly set --strength think they're using 0.75 (mid-tier model) but are actually using 1.0 (most powerful/expensive model). This leads to unexpected API costs.

Fix

Update the help text in pdd/core/cli.py to reflect the actual default of 1.0.

[prompt-driven-github]

🚀 Job Queued!

Job ID: buY7xwzi5yn0BaypwHX4
Triggered by: @jiaminc-cmu
Label: bug

View Live Progress
Model: pdd-opus (claude-opus-4-6)

[prompt-driven-github]

Step 1: Duplicate Check

Status: No duplicates found

Search Performed

• Searched for: strength default, DEFAULT_STRENGTH, 0.75, CLI help text, wrong default, help text incorrect, unexpected API costs strength, cli.py
• Issues reviewed: ~60 (across open and closed issues)

Findings

No existing issue reports the same bug (incorrect default value in the --strength CLI help text). The closest potentially related issues were:

• streamline configuration variable consistency #17 (streamline configuration variable consistency) — an enhancement about ensuring all config variables are settable consistently, not about incorrect help text defaults.
• Missing input validation on numeric CLI options (negative values, out-of-range) #500 (missing input validation on numeric CLI options) — about range validation, not about wrong default values in help text. Already closed.

This is a new, unique issue. Proceeding with investigation.

---

Proceeding to Step 2: Documentation Check

[prompt-driven-github]

Step 2: Documentation Check

Status: Confirmed Bug

Documentation Reviewed

• pdd/__init__.py — source of truth for DEFAULT_STRENGTH
• pdd/core/cli.py — CLI help text for --strength
• README.md — Global Options section
• CHANGELOG.md — historical change notes
• pdd/templates/generic/generate_pddrc_YAML.prompt — pddrc generation template
• docs/ directory — whitepapers, FAQ, prompting guide, tutorials

Findings

The actual default in pdd/__init__.py:12 is:

DEFAULT_STRENGTH = 1.0

However, the documentation and help text reference three different incorrect values across the codebase:

Location	Claimed Default	Actual Default
pdd/__init__.py:12	—	1.0 (source of truth)
pdd/core/cli.py:220 (CLI help text)	0.75	1.0
README.md:604 (Global Options)	0.5	1.0
CHANGELOG.md:1746 (v0.0.80 notes)	0.75	1.0
pdd/templates/generic/generate_pddrc_YAML.prompt	0.75	1.0

This is a clear documentation/code mismatch. The DEFAULT_STRENGTH constant was likely updated from 0.5 → 0.75 → 1.0 over time, but the help text, README, and template were never updated to match.

Impact confirmed: As @Serhan-Asad noted, users who don't explicitly set --strength believe they're using a mid-tier model (0.5 or 0.75) but are actually using the most powerful/expensive model (1.0). The scope is broader than originally reported — the README and pddrc template also have stale values.

---

Proceeding to Step 3: Triage

[prompt-driven-github]

Step 3: Triage

Status: Proceed

Summary

This issue is fully characterized and ready for reproduction. Here's what we know:

Expected behavior: The CLI help text for --strength should accurately reflect the actual default value defined in pdd/__init__.py.

Actual behavior: The help text in pdd/core/cli.py:220 claims the default is 0.75, but the actual constant DEFAULT_STRENGTH at pdd/__init__.py:12 is 1.0. Step 2 also identified two additional stale references (README.md:604 claims 0.5, and the pddrc YAML template claims 0.75).

Reproduction approach: This is a static code inconsistency — reproduction involves comparing the string literal in the CLI help text against the actual DEFAULT_STRENGTH constant. We can verify by:

1. Reading DEFAULT_STRENGTH from pdd/__init__.py (confirmed: 1.0)
2. Reading the help text from pdd/core/cli.py:220 (confirmed: claims 0.75)
3. Checking the two additional locations identified in Step 2

Code areas involved:

• pdd/__init__.py:12 — source of truth (DEFAULT_STRENGTH = 1.0)
• pdd/core/cli.py:220 — CLI help text (says 0.75)
• README.md:604 — documentation (says 0.5)
• pdd/templates/generic/generate_pddrc_YAML.prompt — template (says 0.75)

All values have been directly verified from source. No additional information is needed.

---

Proceeding to Step 4: Reproduction

[prompt-driven-github]

Step 4: Reproduction

Status: Reproduced

Environment

• Python: 3.x (venv at /opt/venv)
• PDD version: 0.0.148
• OS: Linux 6.9.12
• Branch: main (commit 93fa014)

Steps Taken

1. Inspected pdd/__init__.py:12 — confirmed DEFAULT_STRENGTH = 1.0 (source of truth).
2. Ran pdd --help and captured the CLI output.
3. Observed --strength help text reads: "Default: 0.75 or .pddrc value.".
4. Ran python -c "from pdd import DEFAULT_STRENGTH; print(DEFAULT_STRENGTH)" — output: 1.0.
5. Inspected README.md:604 — says "default is 0.5".
6. Inspected pdd/templates/generic/generate_pddrc_YAML.prompt — lines 59, 108, 156 all reference 0.75.

Observed Behavior

CLI help output (actual):

  --strength FLOAT RANGE          Set the strength of the AI model (0.0 to
                                  1.0). Default: 0.75 or .pddrc value.
                                  [0.0<=x<=1.0]

Python import (actual):

>>> from pdd import DEFAULT_STRENGTH
>>> DEFAULT_STRENGTH
1.0

The help text claims 0.75 but the runtime default is 1.0. This is exactly as reported.

Affected Files

• pdd/__init__.py:12 — Source of truth: DEFAULT_STRENGTH = 1.0
• pdd/core/cli.py:220 — Help text hardcodes "Default: 0.75" (should be 1.0)
• README.md:604 — Documentation says "default is 0.5" (should be 1.0)
• pdd/templates/generic/generate_pddrc_YAML.prompt:59,108,156 — Template examples hardcode 0.75 (should be 1.0)

Additional Stale Fallbacks Found

• pdd/server/executor.py:16 — ImportError fallback hardcodes DEFAULT_STRENGTH = 0.5
• pdd/llm_invoke.py:1657 — Function signature hardcodes strength: float = 0.5
• CHANGELOG.md:1746 — References the old 0.5 → 0.75 change, confirming the constant was updated incrementally but docs/help text were not kept in sync

Notes

• The bug is a documentation/help-text staleness issue, not a runtime logic bug. The actual runtime behavior correctly uses DEFAULT_STRENGTH = 1.0 from pdd/__init__.py in all code paths that import the constant.
• However, the stale fallback values in executor.py:16 (0.5) and llm_invoke.py:1657 (0.5) could cause inconsistent behavior if the import fails or those functions are called without explicitly passing strength. These are secondary concerns but worth addressing.
• The issue scope is broader than originally reported — three documentation/template locations plus two code fallbacks all reference outdated values.

---

Proceeding to Step 5: Root Cause Analysis

[prompt-driven-github]

Step 5: Root Cause Analysis

Root Cause Identified: Yes

Summary

The DEFAULT_STRENGTH constant in pdd/__init__.py was incrementally updated from 0.5 → 0.75 → 1.0 over the project's lifetime, but hardcoded string literals in help text, documentation, template files, function signatures, and fallback values were never updated to match. The CHANGELOG explicitly documents the 0.5 → 0.75 transition (v0.0.80, "Default Strength Constant") and the later EXTRACTION_STRENGTH reduction from 0.75 → 0.5 (v0.0.87), but no corresponding updates were made to the scattered hardcoded values. The true root cause is that the codebase has no mechanism to derive user-facing default descriptions from the DEFAULT_STRENGTH constant — each location independently hardcodes a stale numeric literal.

Technical Details

Primary locations with stale values (user-facing):

• Location: pdd/core/cli.py:220

• Problematic Code:

  help="Set the strength of the AI model (0.0 to 1.0). Default: 0.75 or .pddrc value.",

• Why it fails: The help string hardcodes 0.75 instead of 1.0. Users see "Default: 0.75" in pdd --help but actually get 1.0, leading to unexpected use of the most expensive model tier.

• Location: README.md:604

• Problematic Code:

  - `--strength FLOAT`: Set the strength of the AI model (0.0 to 1.0, default is 0.5).

• Why it fails: Documentation hardcodes 0.5 — an even older stale value.

• Location: pdd/templates/generic/generate_pddrc_YAML.prompt:59,108,156

• Problematic Code:

  strength: 0.75  # LLM generation strength (0-1)

• Why it fails: Template for generating .pddrc files uses 0.75 in three places, so auto-generated configs suggest the wrong default.

Secondary locations with stale fallback values (code-level):

File	Line	Stale Value	Should Be
pdd/llm_invoke.py	1657	strength: float = 0.5	DEFAULT_STRENGTH (1.0)
pdd/update_main.py	235, 245	ctx.obj.get("strength", 0.5)	DEFAULT_STRENGTH
pdd/split_main.py	61	ctx.obj.get('strength', 0.5)	DEFAULT_STRENGTH
pdd/server/click_executor.py	168	"strength": 0.5	DEFAULT_STRENGTH
pdd/server/routes/prompts.py	104, 208, 987	Field(0.5, ...)	DEFAULT_STRENGTH
pdd/server/executor.py	16	DEFAULT_STRENGTH = 0.5 (ImportError fallback)	Should match 1.0

Existing Pattern (How Temperature Does It Correctly)

The --temperature option demonstrates the correct pattern — its help text at cli.py:227 says Default: 0.0, which matches DEFAULT_TEMPERATURE = 0.0 in __init__.py. The README at line 612 also correctly states default is 0.0. The --time option similarly matches: help text references DEFAULT_TIME and the README says default is 0.25, matching DEFAULT_TIME = 0.25. Only --strength is inconsistent.

External Research

• Search terms: "click help text dynamic default", "python CLI stale default constant"
• Sources checked: Click framework documentation, GitHub issues
• Relevant findings: This is a common anti-pattern in CLI tools. Click's show_default=True can auto-display the default= value, but PDD uses default=None with show_default=False because the actual default is resolved later through config_resolution.py. The help text string must be manually kept in sync — there is no built-in mechanism to reference a Python constant from within a help string literal. An f-string like help=f"... Default: {DEFAULT_STRENGTH} ..." would solve this programmatically.

Repository History

• Relevant commits: Only one commit exists in the repo snapshot (93fa014 - Bump version), so full git history is unavailable. However, the CHANGELOG provides the historical record.
• CHANGELOG v0.0.80 (line 1746): "Default Strength Constant: Changed hardcoded strength=0.5 default to use DEFAULT_STRENGTH constant (0.75) for consistency" — This was the first consolidation attempt, but it missed the CLI help text, README, template, and several code fallbacks.
• CHANGELOG v0.0.87 (line 1034): "Lower EXTRACTION_STRENGTH from 0.75 to 0.5" — This separated extraction strength from the general default, but the main DEFAULT_STRENGTH was later bumped to 1.0 without updating downstream references.
• CHANGELOG (line 1471): "CLI Strength/Temperature Defaults: Changed CLI --strength and --temperature options to default=None" — This correctly made the CLI defer to config resolution, but the help text string was left with the old 0.75 value.
• Related PRs: No PRs specifically about strength default synchronization were found via gh pr list --search.
• Similar past issues: Issue streamline configuration variable consistency #17 (configuration variable consistency) and Missing input validation on numeric CLI options (negative values, out-of-range) #500 (numeric range validation) are tangentially related but address different problems.

Experiments Performed

1. Verified runtime constant: from pdd import DEFAULT_STRENGTH returns 1.0 — confirmed the source of truth.
2. Verified config resolution: resolve_effective_config() with empty ctx.obj and empty resolved_config correctly falls back to DEFAULT_STRENGTH = 1.0 — the runtime behavior is correct, only the documentation/help text is wrong.
3. Verified llm_invoke signature: inspect.signature(llm_invoke) shows strength: float = 0.5 — the function signature default is stale and would produce wrong behavior if called without passing strength explicitly.
4. Compared temperature pattern: --temperature help text says "Default: 0.0" which matches DEFAULT_TEMPERATURE = 0.0 — confirms this is a strength-specific staleness problem, not a systemic design issue.

Fix Location

• File(s) to modify:

  1. pdd/core/cli.py:220 — Update help string from 0.75 to 1.0 (or use f-string with DEFAULT_STRENGTH)
  2. README.md:604 — Update from 0.5 to 1.0
  3. pdd/templates/generic/generate_pddrc_YAML.prompt:59,108,156 — Update from 0.75 to 1.0
  4. pdd/llm_invoke.py:1657 — Change strength: float = 0.5 to strength: float = DEFAULT_STRENGTH
  5. pdd/update_main.py:235,245 — Change ctx.obj.get("strength", 0.5) to use DEFAULT_STRENGTH
  6. pdd/split_main.py:61 — Change ctx.obj.get('strength', 0.5) to use DEFAULT_STRENGTH
  7. pdd/server/click_executor.py:168 — Change "strength": 0.5 to use DEFAULT_STRENGTH
  8. pdd/server/routes/prompts.py:104,208,987 — Change Field(0.5, ...) to use DEFAULT_STRENGTH
  9. pdd/server/executor.py:16 — Change ImportError fallback from 0.5 to 1.0

• Why this location: The true root cause is not any single file — it's the absence of a single-source-of-truth pattern for the strength default value. The constant exists in pdd/__init__.py:12 but is not referenced by all consumers. Each stale location independently hardcodes an old value, so they must all be updated.

• Existing pattern: The --temperature and --time options already demonstrate the correct approach: help text matches the constant, code uses the imported constant for fallbacks (e.g., ctx.obj.get('time', DEFAULT_TIME) at update_main.py:238), and README states the correct value. The fix should extend this same pattern to all strength references.

• Fix approach:

  1. For help text in cli.py, use an f-string: help=f"Set the strength of the AI model (0.0 to 1.0). Default: {DEFAULT_STRENGTH} or .pddrc value." — this prevents future drift.
  2. For code fallbacks, replace hardcoded 0.5 with DEFAULT_STRENGTH constant (already imported in most files).
  3. For README and template files, update the numeric literals to 1.0.

---

Proceeding to Step 5.5: Prompt Classification

[prompt-driven-github]

Step 5.5: Prompt Classification

Classification: Code Bug + Prompt Defect (mixed)

DEFECT_TYPE: code
PROMPT_FIXED: pdd/templates/generic/generate_pddrc_YAML.prompt

Analysis

This bug affects both code files and a prompt template. The root cause is the same — DEFAULT_STRENGTH was updated from 0.5 → 0.75 → 1.0 over time, but hardcoded literals were never updated to match.

Code bugs (files without corresponding .prompt source-of-truth):

• pdd/core/cli.py:220 — help text hardcodes "Default: 0.75" instead of 1.0
• pdd/llm_invoke.py:1657 — function signature hardcodes strength: float = 0.5
• pdd/server/executor.py:16 — import fallback hardcodes DEFAULT_STRENGTH = 0.5

These are core infrastructure files maintained directly (no PDD prompt generates them). The code simply doesn't match the source of truth constant DEFAULT_STRENGTH = 1.0 in pdd/__init__.py:12.

Prompt defect (prompt template with wrong specification):

• pdd/templates/generic/generate_pddrc_YAML.prompt — specifies strength: 0.75 as the "standard" default in 3 locations (lines 59, 108, 156), contradicting the actual DEFAULT_STRENGTH = 1.0

Evidence

• Source of truth: pdd/__init__.py:12 defines DEFAULT_STRENGTH = 1.0
• Prompt specifies: generate_pddrc_YAML.prompt instructs the LLM to use strength: 0.75 as the standard/recommended default when generating .pddrc files
• Code implements: cli.py shows 0.75, executor.py and llm_invoke.py fall back to 0.5
• User expects: All user-facing values should reflect the actual default of 1.0

Prompt Change Made

File: pdd/templates/generic/generate_pddrc_YAML.prompt

Before (3 locations):

strength: 0.75

After (3 locations):

strength: 1.0

Specific changes:

• Line 59: Example YAML strength: 0.75 → strength: 1.0
• Line 108: Instructions text strength: 0.75 (standard...) → strength: 1.0 (standard...)
• Line 156: Example output strength: 0.75 → strength: 1.0

Conclusion

The prompt defect in generate_pddrc_YAML.prompt has been fixed. The remaining code bugs in cli.py, llm_invoke.py, and executor.py will be addressed via test generation and code fix in subsequent steps. Setting DEFECT_TYPE: code to proceed with the standard test → fix workflow for the code files.

---

Proceeding to Step 6: Test Plan

[prompt-driven-github]

Step 6: Test Plan

Existing Test Coverage

tests/core/test_cli.py (85 tests)

• Framework: pytest with click.testing.CliRunner
• Current coverage: Tests CLI defaults (test_cli_global_options_defaults), explicit options (test_cli_global_options_explicit), help text rendering (test_cli_help), .pddrc shadowing (test_bug_cli_default_strength_shadows_pddrc), and context propagation.
• Gap: No test verifies that the CLI help text for --strength mentions the actual DEFAULT_STRENGTH value. The existing test_cli_help only checks that help renders and contains command names — it does not inspect the default value claimed in the help string.

tests/server/test_executor.py (15 tests)

• Current coverage: Tests create_isolated_context defaults via assert ctx.obj["strength"] == executor_module.DEFAULT_STRENGTH — but this is tautological because it compares the value against the module's own DEFAULT_STRENGTH, which may differ from pdd.DEFAULT_STRENGTH due to the ImportError fallback (hardcoded to 0.5).
• Gap: No test verifies that executor.py's fallback DEFAULT_STRENGTH = 0.5 matches the canonical pdd.DEFAULT_STRENGTH = 1.0.

tests/test_llm_invoke.py (100+ tests)

• Current coverage: Tests model selection, strength-based interpolation, API calls, Pydantic validation.
• Gap: No test verifies that llm_invoke()'s function signature default strength: float = 0.5 matches the canonical pdd.DEFAULT_STRENGTH = 1.0.

Proposed Tests

Test 1: CLI help text displays correct DEFAULT_STRENGTH value

• File: tests/core/test_cli.py (append)
• Input: runner.invoke(cli_command, ["--help"])
• Expected: Help output contains f"Default: {DEFAULT_STRENGTH}" (i.e., "Default: 1.0")
• Actual (before fix): Help output contains "Default: 0.75" — a stale hardcoded string literal at pdd/core/cli.py:220
• Why: This is the primary bug reported in issue CLI help text shows wrong DEFAULT_STRENGTH (0.75 vs actual 1.0) #505. Users read this help text and believe the default is 0.75 when it's actually 1.0.

Test 2: llm_invoke() default strength matches pdd.DEFAULT_STRENGTH

• File: tests/test_llm_invoke.py (append)
• Input: Inspect llm_invoke.__defaults__ or call inspect.signature(llm_invoke) to check the strength parameter default
• Expected: The default value for the strength parameter equals pdd.DEFAULT_STRENGTH (1.0)
• Actual (before fix): Default is 0.5 — a stale hardcoded value at pdd/llm_invoke.py:1657
• Why: This means server-side/programmatic callers of llm_invoke() without an explicit strength get 0.5 instead of the intended 1.0, contradicting the documented default.

Test 3: executor.py fallback DEFAULT_STRENGTH matches canonical constant

• File: tests/server/test_executor.py (append)
• Input: Compare pdd.server.executor.DEFAULT_STRENGTH against pdd.DEFAULT_STRENGTH
• Expected: Both values are equal (1.0)
• Actual (before fix): If the ImportError fallback is triggered, executor.DEFAULT_STRENGTH would be 0.5, not 1.0. Even when the import succeeds, this test ensures the fallback value stays in sync.
• Why: Regression prevention — ensures the fallback never drifts from the canonical constant.

Test 4: Cross-module consistency of DEFAULT_STRENGTH

• File: tests/core/test_cli.py (append)
• Input: Import DEFAULT_STRENGTH from pdd, pdd.server.executor, and check the CLI help string
• Expected: All three sources agree on the same value
• Why: The root cause of this bug was incremental updates to the constant without updating downstream hardcoded references. This test prevents future drift.

Test 5: --strength explicit value still propagated correctly (regression guard)

• File: tests/core/test_cli.py — already covered by test_cli_global_options_explicit and test_cli_explicit_strength_overrides_pddrc
• Status: No new test needed; existing tests already verify this.

Test Location

Test	File	Action
Test 1 (CLI help text)	tests/core/test_cli.py	Append
Test 2 (llm_invoke default)	tests/test_llm_invoke.py	Append
Test 3 (executor fallback)	tests/server/test_executor.py	Append
Test 4 (cross-module consistency)	tests/core/test_cli.py	Append

• Framework: pytest (as used throughout the project)
• Fixtures: runner (from conftest.py) for CLI tests; no special fixtures needed for the others

Notes

1. Test 1 uses a regex/substring match on the help text rather than an exact string comparison, so it won't break if surrounding help text wording changes. The assertion checks f"Default: {DEFAULT_STRENGTH}" using the imported constant, so if DEFAULT_STRENGTH is intentionally changed in the future, the test will still pass — it only fails when the help text hardcodes a different value than the constant.

2. Test 2 uses inspect.signature() to check the function's default parameter value without needing to actually invoke the LLM. This avoids any API calls or complex mocking.

3. Test 3 is a static consistency check — it simply imports the constant from two locations and compares them. This is cheap to run and catches drift immediately.

4. The 0.5 values in the strength interpolation algorithm (lines 1010-1047 of llm_invoke.py) are not stale defaults — they represent the mathematical midpoint in the strength scale and should NOT be changed. Tests should not flag these.

5. Temperature help text already correctly says "Default: 0.0" which matches DEFAULT_TEMPERATURE = 0.0, so no test is needed for that. However, Test 4 could be extended to cover temperature consistency as well for completeness.

---

Proceeding to Step 7: Generate Test

[prompt-driven-github]

Step 7: Generated Test

Test Files

• tests/core/test_cli.py (appended 2 tests)
• tests/test_llm_invoke.py (appended 1 test)
• tests/server/test_executor.py (appended 1 test)

Test Code

Test 1: CLI help text displays correct DEFAULT_STRENGTH value

File: tests/core/test_cli.py

def test_cli_help_shows_correct_default_strength(runner):
    """Issue #505: CLI help text for --strength must display the actual DEFAULT_STRENGTH."""
    result = runner.invoke(cli_command, ["--help"])
    assert result.exit_code == 0
    expected_fragment = f"Default: {DEFAULT_STRENGTH}"
    assert expected_fragment in result.output, (
        f"CLI --help should say '{expected_fragment}' but got:\n{result.output}"
    )
    assert "Default: 0.75" not in result.output, (
        "CLI --help still contains the stale 'Default: 0.75' for --strength"
    )

Test 2: llm_invoke() default strength matches canonical constant

File: tests/test_llm_invoke.py

def test_llm_invoke_default_strength_matches_canonical():
    """Issue #505: llm_invoke() default strength must match pdd.DEFAULT_STRENGTH."""
    import inspect
    import pdd
    sig = inspect.signature(llm_invoke)
    strength_param = sig.parameters["strength"]
    assert strength_param.default == pdd.DEFAULT_STRENGTH, (
        f"llm_invoke() strength default is {strength_param.default}, "
        f"expected pdd.DEFAULT_STRENGTH={pdd.DEFAULT_STRENGTH}"
    )

Test 3: executor.py fallback DEFAULT_STRENGTH matches canonical constant

File: tests/server/test_executor.py

def test_executor_default_strength_matches_canonical():
    """Issue #505: executor.DEFAULT_STRENGTH must match pdd.DEFAULT_STRENGTH."""
    import ast, pdd, inspect
    # Runtime check
    assert executor_module.DEFAULT_STRENGTH == pdd.DEFAULT_STRENGTH
    # Source-level check: hardcoded fallback in except ImportError block
    source = inspect.getsource(executor_module)
    tree = ast.parse(source)
    for node in ast.walk(tree):
        if isinstance(node, ast.ExceptHandler):
            for stmt in ast.walk(node):
                if (isinstance(stmt, ast.Assign)
                        and any(isinstance(t, ast.Name) and t.id == "DEFAULT_STRENGTH"
                                for t in stmt.targets)):
                    value_node = stmt.value
                    if isinstance(value_node, ast.Constant):
                        assert value_node.value == pdd.DEFAULT_STRENGTH, (
                            f"Hardcoded fallback DEFAULT_STRENGTH={value_node.value} "
                            f"does not match pdd.DEFAULT_STRENGTH={pdd.DEFAULT_STRENGTH}"
                        )

Test 4: Cross-module consistency of DEFAULT_STRENGTH

File: tests/core/test_cli.py

def test_default_strength_consistent_across_modules(runner):
    """Issue #505: DEFAULT_STRENGTH must be consistent across all modules."""
    import pdd
    import pdd.server.executor as executor_mod
    assert executor_mod.DEFAULT_STRENGTH == pdd.DEFAULT_STRENGTH
    result = runner.invoke(cli_command, ["--help"])
    assert result.exit_code == 0
    expected_fragment = f"Default: {pdd.DEFAULT_STRENGTH}"
    assert expected_fragment in result.output

What This Test Verifies

These 4 tests catch the bug from 3 angles:

Test	Catches	Failure Message
Test 1	CLI help text says "Default: 0.75" instead of "Default: 1.0"	"Default: 1.0" not in help output
Test 2	llm_invoke() signature defaults to 0.5 instead of 1.0	0.5 != 1.0
Test 3	executor.py ImportError fallback hardcodes 0.5 instead of 1.0	Hardcoded fallback DEFAULT_STRENGTH=0.5 does not match pdd.DEFAULT_STRENGTH=1.0
Test 4	Cross-module drift between pdd, executor, and CLI help	Composite check

Test Results (all 4 FAIL on buggy code ✓)

FAILED tests/core/test_cli.py::test_cli_help_shows_correct_default_strength
  - AssertionError: CLI --help should say 'Default: 1.0' (shows 'Default: 0.75')
FAILED tests/core/test_cli.py::test_default_strength_consistent_across_modules
  - AssertionError: CLI help should contain 'Default: 1.0' (shows 'Default: 0.75')
FAILED tests/test_llm_invoke.py::test_llm_invoke_default_strength_matches_canonical
  - AssertionError: llm_invoke() strength default is 0.5, expected 1.0
FAILED tests/server/test_executor.py::test_executor_default_strength_matches_canonical
  - AssertionError: Hardcoded fallback DEFAULT_STRENGTH=0.5 does not match 1.0

Running the Tests

python -m pytest tests/core/test_cli.py::test_cli_help_shows_correct_default_strength tests/core/test_cli.py::test_default_strength_consistent_across_modules tests/test_llm_invoke.py::test_llm_invoke_default_strength_matches_canonical tests/server/test_executor.py::test_executor_default_strength_matches_canonical -v

---

Proceeding to Step 8: Verification

[prompt-driven-github]

Step 8: Verification Complete

Test Execution

$ python -m pytest tests/core/test_cli.py::test_cli_help_shows_correct_default_strength \
    tests/core/test_cli.py::test_default_strength_consistent_across_modules \
    tests/test_llm_invoke.py::test_llm_invoke_default_strength_matches_canonical \
    tests/server/test_executor.py::test_executor_default_strength_matches_canonical -v

============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.3.5, pluggy-1.6.0
collected 4 items

tests/core/test_cli.py::test_cli_help_shows_correct_default_strength FAILED [ 25%]
tests/core/test_cli.py::test_default_strength_consistent_across_modules FAILED [ 50%]
tests/test_llm_invoke.py::test_llm_invoke_default_strength_matches_canonical FAILED [ 75%]
tests/server/test_executor.py::test_executor_default_strength_matches_canonical FAILED [100%]

FAILURES:
1) test_cli_help_shows_correct_default_strength
   AssertionError: CLI --help should say 'Default: 1.0' but got:
   ...--strength FLOAT RANGE  Set the strength of the AI model (0.0 to 1.0). Default: 0.75 or .pddrc value...

2) test_default_strength_consistent_across_modules
   AssertionError: CLI help should contain 'Default: 1.0' (executor runtime check passed, CLI help text still shows stale 0.75)

3) test_llm_invoke_default_strength_matches_canonical
   AssertionError: llm_invoke() strength default is 0.5, expected pdd.DEFAULT_STRENGTH=1.0

4) test_executor_default_strength_matches_canonical
   AssertionError: Hardcoded fallback DEFAULT_STRENGTH=0.5 in executor.py ImportError handler does not match pdd.DEFAULT_STRENGTH=1.0

============================== 4 failed in 5.34s ===============================

Verification Status

PASS: All 4 tests correctly detect the bug

Each test fails for the right reason — stale hardcoded values that don't match DEFAULT_STRENGTH = 1.0:

Test	Expected	Actual (buggy)	Location
test_cli_help_shows_correct_default_strength	Help text says "Default: 1.0"	"Default: 0.75"	pdd/core/cli.py:220
test_default_strength_consistent_across_modules	CLI help matches canonical constant	CLI help says 0.75	pdd/core/cli.py:220
test_llm_invoke_default_strength_matches_canonical	Function default = 1.0	Function default = 0.5	pdd/llm_invoke.py:1657
test_executor_default_strength_matches_canonical	Fallback = 1.0	Fallback = 0.5	pdd/server/executor.py:16

Summary

Step	Result
Duplicate Check	No duplicates found
Documentation	Confirmed bug — broader than reported (3+ stale locations)
Triage	Proceed — sufficient information to reproduce
Reproduction	Reproduced — pdd --help shows wrong default
Root Cause	Stale hardcoded values after DEFAULT_STRENGTH was updated 0.5 → 0.75 → 1.0
Prompt Classification	Mixed: code bug + prompt defect (prompt already fixed in step 5.5)
Test Plan	4 tests planned across 3 files
Test Generation	4 tests generated, all appended to existing test files
Verification	PASS — All 4 tests fail on buggy code for the correct reasons

Bug Details

• Primary Location: pdd/core/cli.py:220 (help text says 0.75, should be 1.0)
• Secondary Locations: pdd/llm_invoke.py:1657 (default arg 0.5), pdd/server/executor.py:16 (fallback 0.5)
• Root Cause: DEFAULT_STRENGTH constant was incrementally updated from 0.5 → 0.75 → 1.0, but hardcoded string literals and fallback values were never synchronized
• Test Files: tests/core/test_cli.py, tests/test_llm_invoke.py, tests/server/test_executor.py

Next Steps

1. Fix the bug at all identified locations (cli.py help text, llm_invoke.py default arg, executor.py fallback)
2. Run the 4 tests to confirm they pass after the fix
3. Run full test suite to check for regressions
4. Submit PR with fix and tests

---

Proceeding to Step 9: E2E Tests

[prompt-driven-github]

Step 9: E2E Test

Environment Discovery

• Repository language: Python
• Test framework: pytest 8.3.5
• Existing E2E tests found: Yes — 33+ files in tests/test_e2e_*.py (e.g., test_e2e_issue_419_cli_unpushed_commits.py, test_e2e_subprocess_issue_399_ssh_url_message.py)
• Test configuration: pytest.ini (markers: real, integration, slow, e2e)
• Fixtures/helpers used: conftest.py provides runner (CliRunner), create_dummy_files, preserve_pdd_path, restore_standard_streams
• Environment variables required: None for this test (no LLM calls needed)
• Auth/credentials: Not required — this test exercises --help output only

E2E Test File

tests/test_e2e_issue_505_default_strength.py

Test Type

CLI (subprocess-based, like test_e2e_subprocess_issue_399_ssh_url_message.py)

What This Test Verifies

These tests exercise the full user path of running pdd --help via subprocess (exactly as a user would in their terminal) and verify that the --strength option's documented default matches the canonical DEFAULT_STRENGTH constant. Unlike the unit tests from Step 7 (which use CliRunner and inspect internal code), these E2E tests invoke the real CLI binary in a subprocess, exercising the complete import chain, Click option registration, and help text rendering.

3 tests:

1. test_pdd_help_strength_default_matches_canonical — Runs python -m pdd.cli --help and checks the output contains "Default: 1.0" (dynamically read from pdd.DEFAULT_STRENGTH)
2. test_pdd_help_does_not_show_stale_075 — Guards against the specific stale value "Default: 0.75" appearing in help output
3. test_help_default_matches_runtime_config_resolution — Full round-trip: verifies that the help text default, the canonical constant, and the config resolution layer all agree

Test Code

"""
E2E Test (Subprocess-based) for Issue #505: CLI help text shows wrong
DEFAULT_STRENGTH (0.75 vs actual 1.0).

This is a true E2E test that uses subprocess to invoke the actual CLI binary,
exercising the full code path that a user would take.

Bug: When running ``pdd --help``, the ``--strength`` option displays
"Default: 0.75 or .pddrc value" but the actual default used at runtime
(``pdd.DEFAULT_STRENGTH``) is ``1.0``.  Users who rely on the help text
believe they are using a mid-tier model (0.75) but are actually charged
for the most powerful model (1.0).

E2E Test Strategy:
- Use subprocess to run ``python -m pdd.cli --help`` (like a real user)
- Parse the ``--strength`` help text from stdout
- Assert the documented default matches the canonical constant in
  ``pdd/__init__.py``
- Also run ``python -c "from pdd import DEFAULT_STRENGTH; print(DEFAULT_STRENGTH)"``
  to read the canonical value dynamically — no hardcoded expected value

The test should:
- FAIL on the current buggy code (help says 0.75, canonical says 1.0)
- PASS once the bug is fixed (help says 1.0, matching canonical)

Issue: https://github.com/promptdriven/pdd/issues/505
"""

import os
import re
import subprocess
import sys
from pathlib import Path

import pytest


def get_project_root() -> Path:
    """Get the project root directory."""
    current = Path(__file__).parent
    while current != current.parent:
        if (current / "pdd").is_dir() and (current / "pyproject.toml").exists():
            return current
        current = current.parent
    raise RuntimeError("Could not find project root with pdd/ directory")


@pytest.mark.e2e
class TestIssue505E2ESubprocess:
    """
    E2E tests using subprocess to verify the --strength default in CLI help.

    These tests exercise the full CLI path that users take when running
    ``pdd --help`` to check available options and their defaults.
    """

    def _run_pdd_help(self, timeout: int = 30) -> str:
        """Run ``pdd --help`` via subprocess and return combined output."""
        project_root = get_project_root()
        env = os.environ.copy()
        env["PYTHONPATH"] = str(project_root)
        env["PDD_AUTO_UPDATE"] = "false"

        result = subprocess.run(
            [sys.executable, "-m", "pdd.cli", "--help"],
            capture_output=True,
            text=True,
            cwd=str(project_root),
            env=env,
            timeout=timeout,
        )
        return result.stdout + result.stderr

    def _get_canonical_default_strength(self, timeout: int = 10) -> str:
        """Read DEFAULT_STRENGTH from ``pdd/__init__.py`` via subprocess."""
        project_root = get_project_root()
        env = os.environ.copy()
        env["PYTHONPATH"] = str(project_root)

        result = subprocess.run(
            [
                sys.executable, "-c",
                "from pdd import DEFAULT_STRENGTH; print(DEFAULT_STRENGTH)",
            ],
            capture_output=True,
            text=True,
            cwd=str(project_root),
            env=env,
            timeout=timeout,
        )
        assert result.returncode == 0, (
            f"Failed to read DEFAULT_STRENGTH: {result.stderr}"
        )
        return result.stdout.strip()

    def test_pdd_help_strength_default_matches_canonical(self):
        """
        E2E: ``pdd --help`` must show the correct DEFAULT_STRENGTH value.

        User scenario:
        1. User runs ``pdd --help`` to see available options
        2. User reads the --strength option and its documented default
        3. User trusts the help text and does NOT explicitly set --strength

        Expected: Help text says "Default: 1.0" (the canonical value)
        Actual (bug): Help text says "Default: 0.75" (stale value)

        This test FAILS on buggy code, PASSES after fix.
        """
        canonical = self._get_canonical_default_strength()
        help_output = self._run_pdd_help()

        expected_fragment = f"Default: {canonical}"
        assert expected_fragment in help_output, (
            f"BUG DETECTED (Issue #505): CLI --help does not show the correct "
            f"DEFAULT_STRENGTH.\n"
            f"  Expected to find: '{expected_fragment}'\n"
            f"  Canonical DEFAULT_STRENGTH: {canonical}\n\n"
            f"Users see incorrect default and may incur unexpected API costs.\n\n"
            f"Full --help output:\n{help_output}"
        )

    def test_pdd_help_does_not_show_stale_075(self):
        """
        E2E: ``pdd --help`` must NOT claim the strength default is 0.75.

        This guards against the specific stale value reported in the issue.

        This test FAILS on buggy code, PASSES after fix.
        """
        help_output = self._run_pdd_help()

        assert "Default: 0.75" not in help_output, (
            f"BUG DETECTED (Issue #505): CLI --help still contains the stale "
            f"'Default: 0.75' for --strength.\n"
            f"The actual DEFAULT_STRENGTH is 1.0.\n\n"
            f"Full --help output:\n{help_output}"
        )

    def test_help_default_matches_runtime_config_resolution(self):
        """
        E2E: The default shown in ``--help`` must match what the config
        resolution layer actually uses when no ``--strength`` is provided.

        This exercises two separate code paths end-to-end:
        1. CLI help text rendering (``pdd/core/cli.py``)
        2. Config resolution (``pdd/core/config_resolution.py`` →
           ``pdd.DEFAULT_STRENGTH``)

        If these disagree, users are misled about which model tier they use.

        This test FAILS on buggy code, PASSES after fix.
        """
        project_root = get_project_root()
        env = os.environ.copy()
        env["PYTHONPATH"] = str(project_root)
        env.pop("PDD_STRENGTH", None)
        env["PDD_AUTO_UPDATE"] = "false"

        canonical = self._get_canonical_default_strength()

        result = subprocess.run(
            [
                sys.executable, "-c",
                (
                    "import sys, os; "
                    "os.environ.pop('PDD_STRENGTH', None); "
                    "from pdd.core.config_resolution import resolve_strength; "
                    "print(resolve_strength(None, None))"
                ),
            ],
            capture_output=True,
            text=True,
            cwd=str(project_root),
            env=env,
            timeout=10,
        )
        resolved_strength = result.stdout.strip()

        help_output = self._run_pdd_help()

        expected_fragment = f"Default: {canonical}"
        assert expected_fragment in help_output, (
            f"BUG DETECTED (Issue #505): Help text default doesn't match "
            f"canonical DEFAULT_STRENGTH.\n"
            f"  Canonical: {canonical}\n"
            f"  Resolved at runtime: {resolved_strength}\n"
            f"  Help text does not contain '{expected_fragment}'\n\n"
            f"Full --help output:\n{help_output}"
        )

        assert canonical == resolved_strength, (
            f"DEFAULT_STRENGTH ({canonical}) != resolved strength "
            f"({resolved_strength}) — config_resolution disagrees with "
            f"pdd/__init__.py"
        )

Test Execution Result

============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.3.5, pluggy-1.6.0

tests/test_e2e_issue_505_default_strength.py::TestIssue505E2ESubprocess::test_pdd_help_strength_default_matches_canonical FAILED [ 33%]
tests/test_e2e_issue_505_default_strength.py::TestIssue505E2ESubprocess::test_pdd_help_does_not_show_stale_075 FAILED [ 66%]
tests/test_e2e_issue_505_default_strength.py::TestIssue505E2ESubprocess::test_help_default_matches_runtime_config_resolution FAILED [100%]

=================================== FAILURES ===================================
__ TestIssue505E2ESubprocess.test_pdd_help_strength_default_matches_canonical __

>       assert expected_fragment in help_output
E       AssertionError: BUG DETECTED (Issue #505): CLI --help does not show the correct DEFAULT_STRENGTH.
E           Expected to find: 'Default: 1.0'
E           Canonical DEFAULT_STRENGTH: 1.0
E
E         Users see incorrect default and may incur unexpected API costs.
E
E         Full --help output:
E         ...
E           --strength FLOAT RANGE          Set the strength of the AI model (0.0 to
E                                           1.0). Default: 0.75 or .pddrc value.
E                                           [0.0<=x<=1.0]

______ TestIssue505E2ESubprocess.test_pdd_help_does_not_show_stale_075 ________

>       assert "Default: 0.75" not in help_output
E       AssertionError: BUG DETECTED (Issue #505): CLI --help still contains the stale
E         'Default: 0.75' for --strength.
E         The actual DEFAULT_STRENGTH is 1.0.

_ TestIssue505E2ESubprocess.test_help_default_matches_runtime_config_resolution _

>       assert expected_fragment in help_output
E       AssertionError: BUG DETECTED (Issue #505): Help text default doesn't match
E         canonical DEFAULT_STRENGTH.
E           Canonical: 1.0
E           Resolved at runtime: 1.0
E           Help text does not contain 'Default: 1.0'

============================== 3 failed in 18.76s ==============================

E2E Test Status

PASS: E2E test correctly detects the bug

All 3 tests fail on the current buggy code for the correct reason: the CLI help text shows "Default: 0.75" while the canonical DEFAULT_STRENGTH is 1.0 and the config resolution layer also resolves to 1.0. The tests exercise the real subprocess path (python -m pdd.cli --help) with no mocking of the buggy component.

---

Proceeding to Step 10: Create Draft PR