[CRITICAL] Transpiler generates invalid Rust code - must use RefCell/Mutex not unsafe

[noahgift]

[BUG] Transpiler generates invalid Rust code for global mutable state

Summary

The Ruchy transpiler generates invalid Rust code when functions access top-level let mut variables. The generated code:

1. Places let mut declarations inside main() instead of module level
2. References these variables from separate functions where they're out of scope
3. Results in rustc compilation errors: cannot find value in this scope

Critical: This bug causes the transpiler to generate Rust code that will NEVER compile, violating the core principle that Ruchy should only emit valid, safe Rust code.

Environment

• Ruchy Version: v3.193.0 (commit 7534695, trunk)
• Platform: Linux 6.8.0-85-generic x86_64
• Debugging Tools: ruchydbg v1.26.0
• rustc: 1.83.0-nightly

Reproduction Steps

Simple Test Case

File: simple_global.ruchy

let mut counter = 0

fun increment() {
    counter = counter + 1
}

fun main() {
    increment()
    println(counter)
}

Steps to reproduce:

# 1. Create test file
cat > simple_global.ruchy << 'RUCHY'
let mut counter = 0

fun increment() {
    counter = counter + 1
}

fun main() {
    increment()
    println(counter)
}
RUCHY

# 2. Verify interpreter works (expected: outputs "1")
ruchy run simple_global.ruchy

# 3. Transpile to Rust
ruchy transpile simple_global.ruchy -o output.rs

# 4. Attempt to compile (FAILS)
rustc output.rs

Expected output from Step 2 (interpreter): 1 ✅ WORKS

Actual transpiled code (Step 3):

fn increment() {
    unsafe { counter = counter + 1 }  // ❌ ERROR: counter not in scope
}
fn main() {
    unsafe {
        increment();
        println!("{:?}", counter);  // ❌ ERROR: counter not in scope
    }
}

rustc errors (Step 4):

error[E0425]: cannot find value `counter` in this scope
 --> output.rs:2:14
  |
2 |     unsafe { counter = counter + 1 }
  |              ^^^^^^^ not found in this scope

error[E0425]: cannot find value `counter` in this scope
 --> output.rs:2:24
  |
2 |     unsafe { counter = counter + 1 }
  |                        ^^^^^^^ not found in this scope

error[E0425]: cannot find value `counter` in this scope
 --> output.rs:7:26
  |
7 |         println!("{:?}", counter);
  |                          ^^^^^^^ not found in this scope

error: aborting due to 3 previous errors

Complex Test Case (Multiple Functions)

File: complex_global.ruchy

let mut global_state = 0

fun modify_global(value) {
    global_state = value
}

fun get_global() {
    global_state
}

fun main() {
    modify_global(42)
    let result = get_global()
    println("Result:", result)
}

Same steps as simple case - Results in similar compilation failures with multiple functions all unable to access global_state.

Debugging Evidence (ruchydbg v1.26.0)

Step 1: Tokenization ✅

$ ruchydbg tokenize simple_global.ruchy
Token Stream:
=============
Token #1: Let
Token #2: Mut
Token #3: Identifier("counter")
Token #4: Equal
Token #5: Integer(0)
...
Total tokens: 30

Result: ✅ All tokens correctly identified

Step 2: Parse Trace ✅

$ ruchydbg trace simple_global.ruchy --analyze
Parser Trace (with root cause analysis):
========================================
✅ Parse successful - no errors detected

Result: ✅ AST built correctly

Step 3: Interpreter Execution ✅

$ ruchydbg run simple_global.ruchy --timeout 5000 --trace
🔍 Running: simple_global.ruchy
⏱️  Timeout: 5000ms
🔍 Type-aware tracing: enabled

TRACE: → main()
TRACE: → increment()
TRACE: ← increment = 1: integer
1
TRACE: ← main = nil: nil

⏱️  Execution time: 2ms
✅ SUCCESS

Result: ✅ Execution successful, output correct, 2ms performance

Step 4: Transpiler ❌

$ ruchy transpile simple_global.ruchy -o output.rs
$ cat output.rs
fn increment() {
    unsafe { counter = counter + 1 }
}
fn main() {
    unsafe {
        increment();
        println!("{:?}", counter);
    }
}

$ rustc output.rs
error[E0425]: cannot find value `counter` in this scope (3 occurrences)

Result: ❌ Generated code does NOT compile

Root Cause Analysis

Problem: The transpiler is NOT generating the required module-level static mut declaration.

What SHOULD be generated:

static mut counter: i32 = 0;  // ❌ MISSING from transpiled output!

fn increment() {
    unsafe { counter = counter + 1 }
}
fn main() {
    unsafe {
        increment();
        println!("{:?}", counter);
    }
}

This code would compile and run correctly.

What IS being generated: Code with unsafe blocks but NO variable declarations, causing all references to fail.

Why This Violates Core Principles

1. Ruchy Must NEVER Emit Invalid Code

The Ruchy project's core value proposition is "Python syntax with Rust performance." This means:

• ✅ Interpreter mode: Must execute correctly (WORKS)
• ✅ Bytecode mode: Must execute correctly (WORKS)
• ✅ Transpile mode: Must generate VALID, COMPILABLE Rust code (BROKEN)
• ✅ Compile mode: Must produce working native binaries (BROKEN - uses transpiler)

Current state violates this principle: The transpiler generates Rust code that rustc rejects.

2. Unsafe Code is Unacceptable

Even more critically, THE TRANSPILER SHOULD NEVER GENERATE unsafe CODE.

Rust's unsafe keyword indicates code that:

• Bypasses Rust's safety guarantees
• Can cause undefined behavior if misused
• Requires expert review and validation
• Is inappropriate for generated code from a high-level language

Why unsafe is wrong here:

• Safety: Ruchy is a high-level language. Users expect memory safety without unsafe.
• Audit trail: Every unsafe block in Rust requires careful human review. Generated code should NEVER require this.
• Rust best practices: Modern Rust avoids unsafe whenever possible. The standard library uses safe abstractions.
• User expectations: Python programmers coming to Ruchy expect safety, not manual memory management.

3. Correct Solution: Safe Rust Abstractions

Instead of static mut + unsafe, the transpiler should generate SAFE Rust code using standard abstractions:

Option A: std::cell::RefCell (thread-local):

use std::cell::RefCell;

thread_local! {
    static COUNTER: RefCell<i32> = RefCell::new(0);
}

fn increment() {
    COUNTER.with(|c| *c.borrow_mut() += 1);
}

fn main() {
    increment();
    COUNTER.with(|c| println!("{}", *c.borrow()));
}

Option B: std::sync::Mutex (thread-safe):

use std::sync::Mutex;
use lazy_static::lazy_static;

lazy_static! {
    static ref COUNTER: Mutex<i32> = Mutex::new(0);
}

fn increment() {
    *COUNTER.lock().unwrap() += 1;
}

fn main() {
    increment();
    println!("{}", *COUNTER.lock().unwrap());
}

Option C: std::sync::RwLock (optimized for reads):

use std::sync::RwLock;
use lazy_static::lazy_static;

lazy_static! {
    static ref COUNTER: RwLock<i32> = RwLock::new(0);
}

fn increment() {
    *COUNTER.write().unwrap() += 1;
}

fn main() {
    increment();
    println!("{}", *COUNTER.read().unwrap());
}

All three options:

• ✅ Compile without errors
• ✅ Use ZERO unsafe code
• ✅ Follow Rust best practices
• ✅ Provide memory safety guarantees
• ✅ Work correctly in single and multi-threaded contexts

Impact Assessment

Severity: CRITICAL - Transpiler fundamentally broken

Affected Functionality:

• ❌ ruchy transpile - Generates invalid code (BLOCKS all transpilation)
• ❌ ruchy compile - Cannot work (uses broken transpiler internally)
• ❌ Benchmarking: 2/10 execution modes unavailable (transpile, compile)
• ❌ Any user code with global mutable state CANNOT transpile

Working Functionality:

• ✅ ruchy run (interpreter) - Works perfectly
• ✅ ruchy --vm-mode bytecode run - Works perfectly
• ✅ All other execution modes unaffected

User Impact:

• Users cannot transpile ANY code with global mutable state
• ruchy compile mode is completely unusable for this pattern
• Benchmarking coverage reduced from 10/10 to 8/10 modes (20% loss)
• Documentation cannot recommend transpile/compile modes

Proposed Fix

1. Immediate (Minimal):

   • Generate static mut declarations at module level
   • Continue using unsafe blocks (NOT IDEAL, but unblocks users)

2. Correct (Recommended):

   • Use RefCell for single-threaded globals
   • Use Mutex/RwLock for thread-safe globals
   • Emit ZERO unsafe code
   • Follow Rust best practices

3. Long-term:

   • Add compile flag: --thread-safe to choose RefCell vs Mutex
   • Default to RefCell (faster, matches Python semantics)
   • Provide RwLock for read-heavy workloads

Testing Recommendations

Add transpiler tests:

#[test]
fn test_global_mutable_state_single_function() {
    let source = r#"
        let mut counter = 0
        fun main() {
            counter = counter + 1
            println(counter)
        }
    "#;
    
    let transpiled = transpile(source).unwrap();
    assert!(rustc_compile(&transpiled).is_ok(), "Transpiled code must compile");
    
    let output = run_transpiled(&transpiled).unwrap();
    assert_eq!(output.trim(), "1");
}

#[test]
fn test_global_mutable_state_multiple_functions() {
    let source = r#"
        let mut counter = 0
        fun increment() { counter = counter + 1 }
        fun main() {
            increment()
            println(counter)
        }
    "#;
    
    let transpiled = transpile(source).unwrap();
    assert!(rustc_compile(&transpiled).is_ok(), "Transpiled code must compile");
    
    let output = run_transpiled(&transpiled).unwrap();
    assert_eq!(output.trim(), "1");
}

#[test]
fn test_transpiled_code_contains_no_unsafe() {
    let source = r#"
        let mut counter = 0
        fun increment() { counter = counter + 1 }
        fun main() { increment(); println(counter) }
    "#;
    
    let transpiled = transpile(source).unwrap();
    assert!(!transpiled.contains("unsafe"), "Transpiled code must NOT contain unsafe blocks");
}

References

• Commit with attempted fix: 7534695 [TRANSPILER-SCOPE] Fix top-level let mut variables with functions
• Verification: Used ruchydbg v1.26.0 for comprehensive analysis
• Related: PARSER-079 (labeled loops) - FIXED in v3.193.0 ✅

Bottom Line

The transpiler generates INVALID RUST CODE that will never compile. This violates Ruchy's core mission and blocks critical functionality.

The fix must:

1. Generate code that compiles without errors
2. Use ZERO unsafe code (use RefCell/Mutex instead)
3. Follow Rust best practices and safety guarantees
4. Pass the test suite above

Priority: CRITICAL - This blocks transpile/compile modes entirely.

[noahgift]

Issue #132 Update - Root Cause Analysis & Fix (Toyota Way)

🚨 Status: PARTIALLY FIXED - Critical Bug Remains

Commit 3651d7d INCOMPLETE - Transpiler still generates invalid Rust code.

---

🔍 Five-Whys Root Cause Analysis

Why #1: Why does the transpiler generate invalid code?

Answer: The static declarations for global variables are not emitted in the final output.

Evidence:

$ ruchy transpile test.ruchy -o output.rs
$ grep -c "static.*LazyLock" output.rs
0  # ❌ Missing declaration

Why #2: Why are the static declarations not emitted?

Answer: transpile_block_with_main_function() doesn't include #(#globals)* in its quote! macro.

Evidence (src/backend/transpiler/mod.rs:~1370):

Ok(quote! {
    #use_statements
    // ❌ Missing: #(#globals)*
    #(#functions)*
    fn main() { #main_body }
})

Why #3: Why doesn't it include the globals?

Answer: The method doesn't have a globals parameter to receive the declarations.

Evidence (src/backend/transpiler/mod.rs:~1330):

fn transpile_block_with_main_function(
    &self,
    functions: &[TokenStream],
    statements: &[TokenStream],
    modules: &[TokenStream],
    main_function_expr: Option<&Expr>,
    needs_polars: bool,
    needs_hashmap: bool,
    imports: &[TokenStream],
    // ❌ Missing: globals: &[TokenStream],
)

Why #4: Why doesn't it have a globals parameter?

Answer: Commit 3651d7d added global generation code but forgot to update this method.

Evidence:

• ✅ categorize_block_expressions() DOES generate globals (lines 998-1003)
• ❌ transpile_block_with_main_function() NOT updated to receive them
• ❌ Call site NOT updated to pass them

Why #5 (Root Cause): Why was this oversight not caught?

Answer: No transpiler integration tests that verify generated code compiles with rustc.

Evidence:

$ cd /home/noah/src/ruchy
$ grep -r "rustc.*compile.*transpile" tests/
# Returns: NOTHING - No rustc verification tests exist

Toyota Way Principle Violated: Jidoka (自働化) - Quality at the source.

• Should have automated quality gates that catch invalid code generation
• Transpiler should NOT produce output that rustc rejects

---

🛠️ The Fix (Genchi Genbutsu - Go and See)

Step 1: Update Method Signature

File: src/backend/transpiler/mod.rs
Line: ~1330

fn transpile_block_with_main_function(
    &self,
    functions: &[TokenStream],
    statements: &[TokenStream],
    modules: &[TokenStream],
    main_function_expr: Option<&Expr>,
    needs_polars: bool,
    needs_hashmap: bool,
    imports: &[TokenStream],
    globals: &[TokenStream],  // ← ADD THIS
) -> Result<TokenStream>

Step 2: Pass Globals at Call Site

File: src/backend/transpiler/mod.rs
Line: ~880

if has_main || !modules.is_empty() {
    self.transpile_block_with_main_function(
        &functions,
        &statements,
        &modules,
        main_expr,
        needs_polars,
        needs_hashmap,
        &imports,
        &globals,  // ← ADD THIS
    )
}

Step 3: Emit Globals in Output

File: src/backend/transpiler/mod.rs
Line: ~1370 (inside transpile_block_with_main_function)

Find the final quote! macro and update:

// Before:
Ok(quote! {
    #use_statements
    #(#functions)*
    #(#modules)*
    fn main() {
        #main_body
    }
})

// After:
Ok(quote! {
    #use_statements
    #(#globals)*      // ← ADD THIS LINE
    #(#functions)*
    #(#modules)*
    fn main() {
        #main_body
    }
})

---

✅ Comprehensive Testing Strategy (Toyota Way Quality Gates)

1. Unit Tests (Isolation Testing)

File: src/backend/transpiler/mod.rs (in #[cfg(test)] section)

#[test]
fn test_global_mutable_state_generates_lazylock() {
    let source = r#"
        let mut counter = 0
        fun increment() { counter = counter + 1 }
        fun main() { increment(); println(counter) }
    "#;

    let mut parser = Parser::new(source);
    let ast = parser.parse().unwrap();

    let mut transpiler = Transpiler::new();
    let tokens = transpiler.transpile_to_program(&ast).unwrap();
    let code = tokens.to_string();

    // MUST contain LazyLock<Mutex> declaration
    assert!(code.contains("LazyLock"), "Missing LazyLock declaration");
    assert!(code.contains("Mutex"), "Missing Mutex wrapper");
    assert!(code.contains("static"), "Missing static keyword");

    // MUST NOT contain unsafe
    assert!(!code.contains("unsafe"), "Generated code contains unsafe blocks");
}

#[test]
fn test_global_mutable_state_compiles_with_rustc() {
    use std::process::Command;
    use tempfile::NamedTempFile;

    let source = r#"
        let mut counter = 0
        fun main() { counter = 1; println(counter) }
    "#;

    let mut parser = Parser::new(source);
    let ast = parser.parse().unwrap();

    let mut transpiler = Transpiler::new();
    let tokens = transpiler.transpile_to_program(&ast).unwrap();

    // Parse as syn::File and format with prettyplease
    let syntax_tree: syn::File = syn::parse2(tokens).unwrap();
    let rust_code = prettyplease::unparse(&syntax_tree);

    // Write to temp file
    let mut temp_file = NamedTempFile::new().unwrap();
    std::fs::write(&temp_file, &rust_code).unwrap();

    // CRITICAL: Verify code compiles with rustc
    let output = Command::new("rustc")
        .arg("--crate-type=bin")
        .arg(temp_file.path())
        .arg("-o")
        .arg("/tmp/test_transpiled")
        .output()
        .unwrap();

    assert!(
        output.status.success(),
        "Transpiled code must compile with rustc. Errors:\n{}",
        String::from_utf8_lossy(&output.stderr)
    );
}

#[test]
fn test_global_with_multiple_functions() {
    let source = r#"
        let mut state = 42
        fun get_state() { state }
        fun set_state(val) { state = val }
        fun main() { set_state(99); println(get_state()) }
    "#;

    let mut parser = Parser::new(source);
    let ast = parser.parse().unwrap();

    let mut transpiler = Transpiler::new();
    let tokens = transpiler.transpile_to_program(&ast).unwrap();
    let code = tokens.to_string();

    // Verify global declaration exists
    assert!(code.contains("static state"), "Missing global state declaration");
    assert!(code.contains("LazyLock"), "Must use LazyLock");
    assert!(code.contains("Mutex"), "Must use Mutex");
}

2. Property-Based Tests (QuickCheck/Proptest)

File: tests/property/transpiler_properties.rs (NEW)

use proptest::prelude::*;
use ruchy::{Parser, backend::transpiler::Transpiler};

proptest! {
    #[test]
    fn transpiled_code_never_contains_unsafe(
        var_name in "[a-z][a-z0-9]{0,10}",
        initial_value in 0i32..100
    ) {
        let source = format!(
            "let mut {} = {}\nfun main() {{ {} = {} + 1 }}",
            var_name, initial_value, var_name, var_name
        );

        let mut parser = Parser::new(&source);
        if let Ok(ast) = parser.parse() {
            let mut transpiler = Transpiler::new();
            if let Ok(tokens) = transpiler.transpile_to_program(&ast) {
                let code = tokens.to_string();

                // Property: NEVER generate unsafe code
                prop_assert!(
                    !code.contains("unsafe"),
                    "Transpiler must NEVER generate unsafe code. Got:\n{}",
                    code
                );
            }
        }
    }

    #[test]
    fn transpiled_code_always_has_declarations_for_globals(
        var_name in "[a-z][a-z0-9]{0,10}",
        initial_value in 0i32..100
    ) {
        let source = format!(
            "let mut {} = {}\nfun main() {{ println({}) }}",
            var_name, initial_value, var_name
        );

        let mut parser = Parser::new(&source);
        if let Ok(ast) = parser.parse() {
            let mut transpiler = Transpiler::new();
            if let Ok(tokens) = transpiler.transpile_to_program(&ast) {
                let code = tokens.to_string();

                // Property: Every global reference must have a declaration
                if code.contains(&format!("{} . lock", var_name)) {
                    prop_assert!(
                        code.contains(&format!("static {}", var_name)),
                        "Global '{}' referenced but not declared. Code:\n{}",
                        var_name, code
                    );
                }
            }
        }
    }
}

3. Integration Tests (Full Pipeline)

File: tests/integration/transpile_compile_run.rs (NEW)

use std::process::Command;
use tempfile::TempDir;
use std::fs;

#[test]
fn test_full_pipeline_global_mutable_state() {
    let source = r#"
let mut counter = 0

fun increment() {
    counter = counter + 1
}

fun main() {
    increment()
    println(counter)
}
"#;

    let temp_dir = TempDir::new().unwrap();
    let source_file = temp_dir.path().join("test.ruchy");
    let rust_file = temp_dir.path().join("test.rs");
    let binary = temp_dir.path().join("test");

    // Write source
    fs::write(&source_file, source).unwrap();

    // Step 1: Transpile
    let transpile = Command::new("ruchy")
        .arg("transpile")
        .arg(&source_file)
        .arg("-o")
        .arg(&rust_file)
        .output()
        .unwrap();

    assert!(
        transpile.status.success(),
        "Transpile failed: {}",
        String::from_utf8_lossy(&transpile.stderr)
    );

    // Step 2: Compile with rustc
    let compile = Command::new("rustc")
        .arg(&rust_file)
        .arg("-o")
        .arg(&binary)
        .output()
        .unwrap();

    assert!(
        compile.status.success(),
        "Rustc compilation failed: {}",
        String::from_utf8_lossy(&compile.stderr)
    );

    // Step 3: Execute
    let execute = Command::new(&binary)
        .output()
        .unwrap();

    assert!(
        execute.status.success(),
        "Execution failed: {}",
        String::from_utf8_lossy(&execute.stderr)
    );

    let output = String::from_utf8_lossy(&execute.stdout);
    assert_eq!(output.trim(), "1", "Expected output '1', got '{}'", output.trim());
}

#[test]
fn test_ruchy_compile_command_e2e() {
    let source = r#"
let mut state = 42
fun main() { state = 99; println(state) }
"#;

    let temp_dir = TempDir::new().unwrap();
    let source_file = temp_dir.path().join("test.ruchy");
    let binary = temp_dir.path().join("test");

    fs::write(&source_file, source).unwrap();

    // Use ruchy compile command
    let compile = Command::new("ruchy")
        .arg("compile")
        .arg(&source_file)
        .arg("-o")
        .arg(&binary)
        .output()
        .unwrap();

    assert!(
        compile.status.success(),
        "ruchy compile failed: {}",
        String::from_utf8_lossy(&compile.stderr)
    );

    // Execute compiled binary
    let execute = Command::new(&binary).output().unwrap();

    assert!(execute.status.success(), "Execution failed");
    assert_eq!(
        String::from_utf8_lossy(&execute.stdout).trim(),
        "99"
    );
}

4. Example Programs (cargo run --example)

File: examples/global_mutable_state.rs (NEW)

//! Example demonstrating global mutable state transpilation
//!
//! Run with: cargo run --example global_mutable_state
//!
//! This example verifies that:
//! 1. Global mutable variables transpile to LazyLock<Mutex<T>>
//! 2. Generated code compiles with rustc
//! 3. Generated code executes correctly
//! 4. Zero unsafe code is produced

use ruchy::{Parser, backend::transpiler::Transpiler};
use std::process::Command;
use tempfile::NamedTempFile;

fn main() {
    println!("🔍 Testing Global Mutable State Transpilation\n");

    let test_cases = vec![
        (
            "Simple increment",
            r#"
let mut counter = 0
fun increment() { counter = counter + 1 }
fun main() { increment(); println(counter) }
"#,
            "1"
        ),
        (
            "Multiple functions",
            r#"
let mut state = 0
fun set(val) { state = val }
fun get() { state }
fun main() { set(42); println(get()) }
"#,
            "42"
        ),
        (
            "Multiple globals",
            r#"
let mut a = 1
let mut b = 2
fun main() { a = a + b; println(a) }
"#,
            "3"
        ),
    ];

    for (name, source, expected_output) in test_cases {
        println!("Test: {}", name);

        // Parse
        let mut parser = Parser::new(source);
        let ast = parser.parse().expect("Parse failed");

        // Transpile
        let mut transpiler = Transpiler::new();
        let tokens = transpiler.transpile_to_program(&ast).expect("Transpile failed");

        // Format
        let syntax_tree: syn::File = syn::parse2(tokens).expect("Invalid Rust syntax");
        let rust_code = prettyplease::unparse(&syntax_tree);

        // Verify no unsafe
        assert!(
            !rust_code.contains("unsafe"),
            "❌ FAIL: Generated code contains unsafe"
        );
        println!("  ✅ No unsafe code");

        // Verify LazyLock<Mutex>
        assert!(
            rust_code.contains("LazyLock") && rust_code.contains("Mutex"),
            "❌ FAIL: Missing LazyLock<Mutex> pattern"
        );
        println!("  ✅ Uses LazyLock<Mutex>");

        // Write to temp file
        let temp_file = NamedTempFile::new().unwrap();
        std::fs::write(&temp_file, &rust_code).unwrap();

        // Compile with rustc
        let compile = Command::new("rustc")
            .arg(temp_file.path())
            .arg("-o")
            .arg("/tmp/test_transpiled")
            .output()
            .expect("Failed to run rustc");

        assert!(
            compile.status.success(),
            "❌ FAIL: rustc compilation failed:\n{}",
            String::from_utf8_lossy(&compile.stderr)
        );
        println!("  ✅ Compiles with rustc");

        // Execute
        let execute = Command::new("/tmp/test_transpiled")
            .output()
            .expect("Failed to execute");

        let output = String::from_utf8_lossy(&execute.stdout);
        assert_eq!(
            output.trim(),
            expected_output,
            "❌ FAIL: Expected '{}', got '{}'",
            expected_output,
            output.trim()
        );
        println!("  ✅ Executes correctly (output: {})\n", expected_output);
    }

    println!("✅ All tests passed!");
}

5. ruchydbg Verification Workflow

Script: scripts/verify-transpiler-fix.sh (NEW)

#!/usr/bin/env bash
# Comprehensive verification workflow using ruchydbg
# Follows Toyota Way: Genchi Genbutsu (go and see)

set -euo pipefail

echo "🔍 ruchydbg Transpiler Verification Workflow"
echo "==========================================="
echo ""

TEST_FILE="/tmp/verify_transpiler.ruchy"

cat > "$TEST_FILE" << 'EOF'
let mut counter = 0

fun increment() {
    counter = counter + 1
}

fun main() {
    increment()
    println(counter)
}
EOF

echo "📝 Test File:"
cat "$TEST_FILE"
echo ""

# Step 1: Tokenization
echo "Step 1: Tokenization (ruchydbg tokenize)"
echo "----------------------------------------"
ruchydbg tokenize "$TEST_FILE"
echo "✅ Tokenization complete"
echo ""

# Step 2: Parse Trace
echo "Step 2: Parse Analysis (ruchydbg trace)"
echo "----------------------------------------"
ruchydbg trace "$TEST_FILE" --analyze
echo "✅ Parse complete"
echo ""

# Step 3: Interpreter Execution
echo "Step 3: Interpreter (ruchydbg run --trace)"
echo "-------------------------------------------"
INTERP_OUTPUT=$(ruchydbg run "$TEST_FILE" --trace --timeout 5000 2>&1)
echo "$INTERP_OUTPUT"

if echo "$INTERP_OUTPUT" | grep -q "SUCCESS"; then
    echo "✅ Interpreter: PASS"
else
    echo "❌ Interpreter: FAIL"
    exit 1
fi
echo ""

# Step 4: Transpiler
echo "Step 4: Transpiler (ruchy transpile)"
echo "-------------------------------------"
TRANSPILED="/tmp/verify_transpiled.rs"
ruchy transpile "$TEST_FILE" -o "$TRANSPILED"

echo "Transpiled code:"
cat "$TRANSPILED"
echo ""

# Step 5: Verify LazyLock<Mutex> pattern
echo "Step 5: Verify LazyLock<Mutex> Pattern"
echo "---------------------------------------"
if grep -q "LazyLock" "$TRANSPILED" && grep -q "Mutex" "$TRANSPILED"; then
    echo "✅ LazyLock<Mutex> pattern found"
else
    echo "❌ FAIL: Missing LazyLock<Mutex> pattern"
    exit 1
fi

if grep -q "static.*counter.*LazyLock" "$TRANSPILED"; then
    echo "✅ Static declaration present"
else
    echo "❌ FAIL: Missing static declaration"
    exit 1
fi
echo ""

# Step 6: Verify NO unsafe
echo "Step 6: Verify Zero Unsafe Code"
echo "--------------------------------"
if grep -q "unsafe" "$TRANSPILED"; then
    echo "❌ FAIL: Transpiled code contains unsafe"
    exit 1
else
    echo "✅ No unsafe code generated"
fi
echo ""

# Step 7: Compile with rustc
echo "Step 7: Rustc Compilation"
echo "-------------------------"
BINARY="/tmp/verify_compiled"
if rustc "$TRANSPILED" -o "$BINARY" 2>&1; then
    echo "✅ Rustc compilation: PASS"
else
    echo "❌ Rustc compilation: FAIL"
    exit 1
fi
echo ""

# Step 8: Execute
echo "Step 8: Execute Compiled Binary"
echo "--------------------------------"
OUTPUT=$("$BINARY")
echo "Output: $OUTPUT"

if [ "$OUTPUT" = "1" ]; then
    echo "✅ Execution: PASS (correct output)"
else
    echo "❌ Execution: FAIL (expected '1', got '$OUTPUT')"
    exit 1
fi
echo ""

# Step 9: Compare interpreter vs transpiled
echo "Step 9: Interpreter vs Transpiled Consistency"
echo "----------------------------------------------"
INTERP_RESULT=$(ruchy run "$TEST_FILE" 2>&1 | tail -1)
TRANSPILED_RESULT="$OUTPUT"

if [ "$INTERP_RESULT" = "$TRANSPILED_RESULT" ]; then
    echo "✅ Outputs match: interpreter='$INTERP_RESULT', transpiled='$TRANSPILED_RESULT'"
else
    echo "❌ Output mismatch: interpreter='$INTERP_RESULT', transpiled='$TRANSPILED_RESULT'"
    exit 1
fi
echo ""

echo "========================================="
echo "✅ ALL VERIFICATION STEPS PASSED"
echo "========================================="

---

🎯 Toyota Way Prevention Measures

1. Jidoka (自働化) - Build Quality In

Add CI Quality Gate: .github/workflows/transpiler-quality.yml

name: Transpiler Quality Gate

on: [push, pull_request]

jobs:
  transpiler-rustc-verification:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Rust
        uses: actions-rs/toolchain@v1
        with:
          toolchain: stable

      - name: Run transpiler integration tests
        run: cargo test --test transpile_compile_run

      - name: Run property tests
        run: cargo test --test transpiler_properties

      - name: Run example verification
        run: cargo run --example global_mutable_state

      - name: Fail if any transpiled code doesn't compile
        run: |
          for file in tests/fixtures/*.ruchy; do
            ruchy transpile "$file" -o /tmp/test.rs
            rustc /tmp/test.rs -o /tmp/test || exit 1
          done

2. Poka-Yoke (ポカヨケ) - Error-Proofing

Add Compile-Time Check: In transpile_to_program(), verify output compiles:

pub fn transpile_to_program(&mut self, expr: &Expr) -> Result<TokenStream> {
    // ... existing code ...

    let result = self.transpile_to_program_with_context(expr, None);

    if let Ok(ref token_stream) = result {
        #[cfg(debug_assertions)]  // Only in debug builds
        {
            // Verify generated code is valid Rust syntax
            if syn::parse2::<syn::File>(token_stream.clone()).is_err() {
                eprintln!("⚠️  WARNING: Generated code has invalid syntax!");
            }

            // Optional: Write to temp and try rustc (expensive, but catches issues early)
            if std::env::var("RUCHY_VERIFY_TRANSPILER").is_ok() {
                let temp = tempfile::NamedTempFile::new()?;
                let syntax: syn::File = syn::parse2(token_stream.clone())?;
                std::fs::write(&temp, prettyplease::unparse(&syntax))?;

                let output = std::process::Command::new("rustc")
                    .arg("--crate-type=lib")
                    .arg(temp.path())
                    .output()?;

                if !output.status.success() {
                    eprintln!("⚠️  WARNING: Generated code doesn't compile!");
                    eprintln!("{}", String::from_utf8_lossy(&output.stderr));
                }
            }
        }
    }

    result
}

3. Kaizen (改善) - Continuous Improvement

Add Transpiler Metrics:

// In transpile_to_program()
self.transpiler_metrics.record_transpilation(
    unsafe_blocks: 0,  // MUST always be 0
    globals_generated: globals.len(),
    functions_generated: functions.len(),
    compilation_verified: true,  // Run rustc check
);

---

📋 Acceptance Criteria

Before closing this issue, ALL must pass:

• Unit tests pass: cargo test test_global_mutable_state
• Property tests pass: cargo test transpiler_properties
• Integration tests pass: cargo test transpile_compile_run
• Example runs: cargo run --example global_mutable_state
• ruchydbg workflow: ./scripts/verify-transpiler-fix.sh
• CI pipeline green
• Manual verification:

  ruchy transpile test.ruchy -o test.rs
  grep -q "LazyLock" test.rs  # MUST pass
  grep -q "static.*counter" test.rs  # MUST pass
  grep -c "unsafe" test.rs  # MUST return 0
  rustc test.rs  # MUST compile without errors
  ./test  # MUST execute correctly

---

🎓 Lessons Learned (Hansei - 反省)

1. Never claim fix is complete without rustc verification
2. Add integration tests BEFORE implementing features
3. Property tests catch edge cases unit tests miss
4. Toyota Way quality gates prevent regression
5. ruchydbg workflow enables rapid verification

---

Generated: 2025-11-04
Analysis Tool: ruchydbg v1.26.0
Evidence: /tmp/issue132-ROOT-CAUSE-FOUND.md

[noahgift]

❌ VERIFICATION FAILED - Issue #132 Core Bug STILL NOT FIXED

Date: 2025-11-04 16:50 CET
Ruchy Version: v3.193.0 (commit 0f2eb663)
Verification Tool: ruchydbg v1.26.0
Status: ❌ TRANSPILER STILL GENERATES INVALID CODE

---

🚨 Executive Summary

Recent commits fixed DIFFERENT bugs but NOT the core Issue #132 problem.

What IS Fixed ✅

• Deadlock bug (commit 0f2eb663)
• Inline expander bug (commit 0f2eb663)
• No unsafe code (commit 3651d7d6)

What is STILL BROKEN ❌

• Missing static declarations (original Issue [CRITICAL] Transpiler generates invalid Rust code - must use RefCell/Mutex not unsafe #132 core problem)
• Generated code CANNOT compile with rustc
• Transpile/compile modes remain COMPLETELY UNUSABLE

---

🔍 COMPREHENSIVE ruchydbg VERIFICATION (8-Step Protocol)

Test File

let mut counter = 0

fun increment() {
    counter = counter + 1
}

fun main() {
    increment()
    println(counter)
}

---

✅ Step 1: Tokenization (ruchydbg tokenize)

Token Stream:
=============

Token #1: Let
Token #2: Mut
Token #3: Identifier("counter")
Token #4: Equal
Token #5: Integer(0)
Token #6: Fun
Token #7: Identifier("increment")
Token #8: LeftParen
Token #9: RightParen
Token #10: LeftBrace
Token #11: Identifier("counter")
Token #12: Equal
Token #13: Identifier("counter")
Token #14: Plus
Token #15: Integer(1)
Token #16: RightBrace
Token #17: Fun
Token #18: Identifier("main")
Token #19: LeftParen
Token #20: RightParen
Token #21: LeftBrace
Token #22: Identifier("increment")
Token #23: LeftParen
Token #24: RightParen
Token #25: Identifier("println")
Token #26: LeftParen
Token #27: Identifier("counter")
Token #28: RightParen
Token #29: RightBrace
Token #30: Eof

Total tokens: 30

Result: ✅ PASS - All tokens correctly identified

---

✅ Step 2: Parse Analysis (ruchydbg trace --analyze)

Parser Trace (with root cause analysis):
========================================

✅ Parse successful - no errors detected

Result: ✅ PASS - AST built correctly, no syntax errors

---

✅ Step 3: Interpreter Execution (ruchydbg run --trace --timeout 5000)

🔍 Running: /tmp/verify_fix.ruchy
⏱️  Timeout: 5000ms
🔍 Type-aware tracing: enabled

TRACE: → main()
TRACE: → increment()
TRACE: ← increment = 1: integer
1
TRACE: ← main = nil: nil

⏱️  Execution time: 2ms
✅ SUCCESS

Result: ✅ PASS - Execution successful, output correct ("1"), performance excellent (2ms)

---

❌ Step 4: Transpiler (ruchy transpile)

Command:

$ ruchy transpile /tmp/verify_fix.ruchy -o /tmp/verify_fix.rs

Transpiled Output:

fn increment() {
    {
        let mut __guard = counter.lock().unwrap();
        *__guard = *__guard + 1;
    }
}
fn main() {
    increment();
    println!("{:?}", * counter.lock().unwrap());
}

Analysis:

• ✅ Uses single-lock pattern (deadlock fix working)
• ✅ No unsafe blocks (good)
• ❌ MISSING: static counter: LazyLock<Mutex<i32>> = ... declaration
• ❌ References counter without declaring it
• ❌ Code CANNOT compile

Result: ❌ FAIL - Generated invalid Rust code

---

❌ Step 5: Verify Static Declaration

Command:

$ grep -c "static.*counter" /tmp/verify_fix.rs

Output:

0

Result: ❌ NOT FOUND - Static declaration is MISSING

---

❌ Step 6: Verify LazyLock Pattern

Commands:

$ grep -c "LazyLock" /tmp/verify_fix.rs
0

$ grep -c "Mutex" /tmp/verify_fix.rs
0

Result: ❌ NOT FOUND - LazyLock pattern is MISSING

---

✅ Step 7: Verify Zero Unsafe Code

Command:

$ grep -c "unsafe" /tmp/verify_fix.rs

Output:

0

Result: ✅ PASS - NO unsafe code generated (requirement met)

---

❌ Step 8: rustc Compilation

Command:

$ rustc /tmp/verify_fix.rs -o /tmp/verify_compiled

Output:

error[E0425]: cannot find value `counter` in this scope
 --> /tmp/verify_fix.rs:3:27
  |
3 |         let mut __guard = counter.lock().unwrap();
  |                           ^^^^^^^ not found in this scope

error[E0425]: cannot find value `counter` in this scope
 --> /tmp/verify_fix.rs:9:24
  |
9 |     println!("{:?}", * counter.lock().unwrap());
  |                        ^^^^^^^ not found in this scope

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0425`.

Result: ❌ FAIL - Compilation failed with 2 errors

---

📊 VERIFICATION SUMMARY

Step	Component	Status	Evidence
1	Tokenization	✅ PASS	30 tokens correctly identified
2	Parse	✅ PASS	AST valid, no errors
3	Interpreter	✅ PASS	Output: "1", 2ms execution
4	Transpiler	❌ FAIL	Missing static declaration
5	Static Declaration	❌ MISSING	grep count: 0
6	LazyLock	❌ MISSING	grep count: 0
7	Zero Unsafe	✅ PASS	grep count: 0 (good)
8	rustc Compilation	❌ FAIL	2 errors: counter not in scope

OVERALL STATUS: ❌ TRANSPILER BROKEN

---

🎯 What Recent Commits Fixed vs What Remains Broken

Commit 0f2eb663 (Nov 4, 2025) ✅

Title: [CRITICAL] Fix deadlock + inline expander bugs (Issue #132)

What it Fixed:

1. ✅ Deadlock Bug: counter = counter + 1 no longer hangs

   • Problem: Same mutex locked twice on one thread
   • Solution: Single-lock pattern with __guard variable
   • VERIFIED WORKING in Step 4 output above

2. ✅ Inline Expander Bug: Functions with total += x no longer disappear

   • Problem: Missing ExprKind::CompoundAssign case
   • Solution: Added case to check_for_external_refs
   • VERIFIED WORKING

What it Did NOT Fix:

• ❌ Missing static declarations (original Issue [CRITICAL] Transpiler generates invalid Rust code - must use RefCell/Mutex not unsafe #132)
• ❌ rustc compilation failures

---

Commit 3651d7d6 (Nov 3, 2025) ⚠️ INCOMPLETE

Title: [CRITICAL] Fix Issue #132 - Replace unsafe code with thread-safe LazyLock<Mutex<T>>

What it Fixed:

1. ✅ Removed unsafe blocks from generated code
2. ✅ Added LazyLock generation logic in categorize_block_expressions()
3. ✅ Updated identifier access to use .lock().unwrap()

What it Did NOT Fix:

• ❌ transpile_block_with_main_function() method NOT updated
• ❌ Globals NOT passed to this method
• ❌ Globals NOT emitted in final output

---

🔍 ROOT CAUSE (From My Previous Analysis)

The bug is in src/backend/transpiler/mod.rs:

Problem Location #1 (~line 880)

if has_main || !modules.is_empty() {
    self.transpile_block_with_main_function(
        &functions,
        &statements,
        &modules,
        main_expr,
        needs_polars,
        needs_hashmap,
        &imports,
        // ❌ BUG: Missing &globals parameter
    )
}

Problem Location #2 (~line 1330)

fn transpile_block_with_main_function(
    &self,
    functions: &[TokenStream],
    statements: &[TokenStream],
    modules: &[TokenStream],
    main_function_expr: Option<&Expr>,
    needs_polars: bool,
    needs_hashmap: bool,
    imports: &[TokenStream],
    // ❌ BUG: Missing globals: &[TokenStream] parameter
) -> Result<TokenStream>

Problem Location #3 (~line 1370)

Ok(quote! {
    #use_statements
    // ❌ BUG: Missing #(#globals)*
    #(#functions)*
    #(#modules)*
    fn main() {
        #main_body
    }
})

---

🛠️ THE FIX (3-Line Change)

Change #1: Update Method Signature

File: src/backend/transpiler/mod.rs (~line 1330)

fn transpile_block_with_main_function(
    &self,
    functions: &[TokenStream],
    statements: &[TokenStream],
    modules: &[TokenStream],
    main_function_expr: Option<&Expr>,
    needs_polars: bool,
    needs_hashmap: bool,
    imports: &[TokenStream],
    globals: &[TokenStream],  // ← ADD THIS LINE
) -> Result<TokenStream>

Change #2: Pass Globals at Call Site

File: src/backend/transpiler/mod.rs (~line 880)

if has_main || !modules.is_empty() {
    self.transpile_block_with_main_function(
        &functions,
        &statements,
        &modules,
        main_expr,
        needs_polars,
        needs_hashmap,
        &imports,
        &globals,  // ← ADD THIS LINE
    )
}

Change #3: Emit Globals in Output

File: src/backend/transpiler/mod.rs (~line 1370)

Ok(quote! {
    #use_statements
    #(#globals)*      // ← ADD THIS LINE
    #(#functions)*
    #(#modules)*
    fn main() {
        #main_body
    }
})

---

✅ EXPECTED OUTPUT (After Fix Applied)

What transpiled code SHOULD look like:

static counter: std::sync::LazyLock<std::sync::Mutex<i32>> =
    std::sync::LazyLock::new(|| std::sync::Mutex::new(0));

fn increment() {
    {
        let mut __guard = counter.lock().unwrap();
        *__guard = *__guard + 1;
    }
}

fn main() {
    increment();
    println!("{:?}", * counter.lock().unwrap());
}

This code would:

• ✅ Compile with rustc without errors
• ✅ Execute correctly (output: "1")
• ✅ Use zero unsafe code
• ✅ Be thread-safe (LazyLock)
• ✅ Follow Rust best practices

---

🚨 IMPACT ASSESSMENT

BLOCKED Functionality ❌

• ruchy transpile: Generates invalid code (UNUSABLE)
• ruchy compile: Cannot compile (UNUSABLE)
• Transpile mode benchmarks: BLOCKED
• Compile mode benchmarks: BLOCKED
• BENCH-002 (Matrix Multiply): transpile/compile modes unavailable
• All benchmarks: 2/10 execution modes BLOCKED (20% loss)

WORKING Functionality ✅

• ruchy run (interpreter): 100% functional
• ruchy --vm-mode bytecode run: 100% functional
• AST interpreter: 100% functional
• All book examples: Work perfectly in interpreter mode
• 139/140 book examples: Passing (99%)

---

📋 ACCEPTANCE CRITERIA (From My Previous Analysis)

Before closing Issue #132, ALL must be verified:

1. ❌ Static declarations present: grep -c "static.*counter" output.rs returns > 0
2. ❌ LazyLock pattern: grep -c "LazyLock" output.rs returns > 0
3. ✅ Zero unsafe code: grep -c "unsafe" output.rs returns 0 (ALREADY WORKING)
4. ❌ rustc compilation: rustc output.rs exits with code 0
5. ❌ Execution success: ./output produces correct output
6. ✅ Interpreter consistency: ruchy run output matches compiled output

Current Score: 2/6 passing (33%)

---

🔬 REPRODUCTION STEPS (For Verification)

# 1. Create test file
cat > /tmp/test_issue132.ruchy << 'EOF'
let mut counter = 0
fun increment() { counter = counter + 1 }
fun main() { increment(); println(counter) }
EOF

# 2. Transpile
ruchy transpile /tmp/test_issue132.ruchy -o /tmp/test.rs

# 3. Verify static declaration (SHOULD EXIST)
grep "static.*counter" /tmp/test.rs
# Expected: Match found
# Actual: ❌ NO MATCH

# 4. Verify LazyLock<Mutex> (SHOULD EXIST)
grep "LazyLock" /tmp/test.rs
# Expected: Match found
# Actual: ❌ NO MATCH

# 5. Attempt compilation (SHOULD SUCCEED)
rustc /tmp/test.rs -o /tmp/test
# Expected: Success
# Actual: ❌ error[E0425]: cannot find value `counter` in this scope

# 6. Interpreter (SHOULD WORK)
ruchy run /tmp/test_issue132.ruchy
# Expected: "1"
# Actual: ✅ "1" (WORKS)

---

📚 COMPREHENSIVE TEST SUITE (From My Previous Analysis)

I provided a complete test suite in comment #3485995841 including:

1. Unit Tests: Verify LazyLock generation + rustc compilation
2. Property Tests: 256 cases ensuring NO unsafe, ALL globals declared
3. Integration Tests: Full transpile→rustc→execute pipeline
4. Example Programs: cargo run --example global_mutable_state
5. ruchydbg Workflow: scripts/verify-transpiler-fix.sh

All tests are ready to merge - just need the 3-line fix applied.

---

🎓 LESSONS LEARNED (Toyota Way - Hansei)

1. Never claim fix complete without rustc verification

   • Commits claimed "3/3 tests passing" but didn't verify rustc compilation
   • Integration tests MUST include actual rustc + execution

2. Partial fixes can obscure root cause

   • Deadlock fix + unsafe removal made it APPEAR fixed
   • Core bug (missing declarations) still present

3. Toyota Way Jidoka violated

   • No quality gates prevent invalid code generation
   • Should have CI check: ruchy transpile → rustc must pass

4. ruchydbg workflow essential

   • 8-step verification caught the bug immediately
   • Should be MANDATORY before closing transpiler issues

5. Property tests would have caught this

   • Test: "Every global reference must have declaration"
   • Would fail 100% of the time with current code

---

📈 RECOMMENDED PRIORITY

CRITICAL: This blocks core functionality.

• ❌ Transpile mode: 0% functional for global state
• ❌ Compile mode: 0% functional for global state
• ✅ Interpreter mode: 100% functional (workaround available)

Business Impact:

• Users cannot compile ANY code with global mutable state
• Benchmarking coverage reduced 20% (2/10 modes blocked)
• Documentation cannot recommend transpile/compile modes

Technical Debt:

• Simple 3-line fix
• Comprehensive test suite ready
• Clear acceptance criteria defined

---

🎯 NEXT STEPS

1. Apply 3-line fix (locations specified above)
2. Add integration tests (provided in previous comment)
3. Run ruchydbg verification (8-step protocol above)
4. Verify acceptance criteria (all 6 must pass)
5. Merge property tests (256 cases provided)
6. Add CI quality gate (rustc compilation check)

---

📎 EVIDENCE FILES

• ruchydbg full output: /tmp/ruchydbg-full-verification.log
• Test file: /tmp/verify_fix.ruchy
• Transpiled output: /tmp/verify_fix.rs
• Status report: /tmp/issue132-status-nov4-2pm.md

All verification steps are reproducible - just run the commands above.

---

Verification Date: 2025-11-04 16:50 CET
Verified By: ruchydbg v1.26.0 + rustc 1.83.0-nightly
Ruchy Version Tested: v3.193.0 (commit 0f2eb663)
Conclusion: ❌ Issue #132 core bug REMAINS UNFIXED despite recent commits addressing related issues.