Executing sqlite statements inside model init function crashes the program

[Godrak]

I used to initialize DB tables in the init function, which now segfaults the program in the newest version of webserver.
Modified code from examples:

init! : {} => Result Model [Exit I32 Str]_
init! = |{}|
    db_path = read_env_var!("DB_PATH")?
    list_todos_stmt = prepare_stmt!(db_path, "SELECT id, task, status FROM todos")?
    create_todo_stmt = prepare_stmt!(db_path, "INSERT INTO todos (task, status) VALUES (:task, :status)")?
    last_created_todo_stmt = prepare_stmt!(db_path, "SELECT id, task, status FROM todos WHERE id = last_insert_rowid()")?
    begin_stmt = prepare_stmt!(db_path, "BEGIN")?
    end_stmt = prepare_stmt!(db_path, "END")?
    rollback_stmt = prepare_stmt!(db_path, "ROLLBACK")?

    Sqlite.execute_prepared!({ bindings: [], stmt: list_todos_stmt })?   **<--  this will crash**

    Ok({ list_todos_stmt, create_todo_stmt, last_created_todo_stmt, begin_stmt, end_stmt, rollback_stmt })

[Anton-4]

Thanks for reporting @Godrak, I'll try to investigate this week.

[Godrak]

Hi, thanks. I want to provide some additional info.
It seems that the issue is more 'widespread', I found out my project will crash no matter where I call the sql query/execute, even in the respond function.

But I also tested the provided todos example, and it crashed only when I placed it into init 🤷

regarding the crashes, I am only getting two errors:
munmap_chunk(): invalid pointer
or
free(): invalid pointer

[Anton-4]

Thanks for the info, can you share the output of the valgrind command:

roc build main.roc --linker=legacy
valgrind --leak-check=full --track-origins=yes ./main

[Godrak]

` main
==369164== Memcheck, a memory error detector
==369164== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==369164== Using Valgrind-3.23.0 and LibVEX; rerun with -h for copyright info
==369164== Command: ./main
==369164==
[main.roc:41] "Starting init function"
[main.roc:49] "Checked monitor flag existence"
no monitor flag found
[main.roc:57] "About to parse user data"
[main.roc:73] "Parsing user from line: $line"
[main.roc:78] "User parsed successfully: $name"
[main.roc:73] "Parsing user from line: $line"
[main.roc:78] "User parsed successfully: $name"
[main.roc:73] "Parsing user from line: $line"
[main.roc:78] "User parsed successfully: $name"
[main.roc:63] "User data parsed successfully"
[main.roc:66] "Sender created from first user"
[main.roc:68] "Init completed successfully"
Listening on http://127.0.0.1:8000
[main.roc:100] "Starting respond! function for URI: $req.uri"
2025-03-25T19:56:29Z GET /
[main.roc:108] "URL path parsed: $split_url"
==369164== Thread 18 tokio-runtime-w:
==369164== Invalid free() / delete / delete[] / realloc()
==369164== at 0x4848BD4: free (in /nix/store/jlcmn1sc1k1ryzk2lzxs3af4m1ghmdy6-valgrind-3.23.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==369164== by 0x142205: ??? (in /home/pavel/trunk/roc/kodos/main)
==369164== by 0x2A1EAC: ??? (roc_app:0)
==369164== by 0x289CFF: Host_sqlite_reset!_655e3b1e4bc545f728afe7d233602ad4e5fa8c6075c7dbd091b3a5a8306124 (roc_app:0)
==369164== by 0x2399D1: Sqlite_reset!_a5b4f8c666254e193194f24d1dd3b7c2933e66f2ac1fd2083391a2fc5636c1e (roc_app:0)
==369164== by 0x19B4B6: Sqlite_execute_prepared!_e446f1959ee19499c2da2a9193eadbbb9d88e146dec44dd7a0771d11f52a2 (roc_app:0)
==369164== by 0x1FF623: Sqlite_execute!_eee16dba3c658eecc5d993336fe47b6ebac6f00b18a081a228b54c52306f (roc_app:0)
==369164== by 0x21EFB9: #UserApp_respond!_4eaf7d11eede901687feecdb5946715fbfee5e6384a979b6615e57e376fe (in /home/pavel/trunk/roc/kodos/main)
==369164== by 0x1A6C16: _respond_for_host!_7d9d8bd1d2df1e453bff8782a93227a30a84d615dd2adb918f6b8ac67a8b795 (roc_app:0)
==369164== by 0x1A6F4C: roc__respond_for_host_1_exposed_generic (roc_app:0)
==369164== by 0x39D2D3: tokio::runtime::task::raw::poll (in /home/pavel/trunk/roc/kodos/main)
==369164== by 0x41E29D: std::sys_common::backtrace::__rust_begin_short_backtrace (in /home/pavel/trunk/roc/kodos/main)
==369164== Address 0x4f78000 is in a rw- anonymous segment
==369164==
Server error:
UnhandledRows

Tip: If you do not want to see this error, use Result.map_err to handle the error.
Docs for Result.map_err: https://www.roc-lang.org/builtins/Result#map_err
[main.roc:100] "Starting respond! function for URI: $req.uri"
2025-03-25T19:56:30Z GET /favicon.ico
[main.roc:108] "URL path parsed: $split_url"
Server error:
UnhandledRows

Tip: If you do not want to see this error, use Result.map_err to handle the error.
Docs for Result.map_err: https://www.roc-lang.org/builtins/Result#map_err`

[Anton-4]

Thanks @Godrak I'll try to reproduce and investigate.

[Anton-4]

Alright, so in case you want to return rows from the db you need to use query_many_prepared instead of execute_prepared:

init! : {} => Result Model _
init! = |{}|
    # Read DB_PATH environment variable
    db_path = Env.var!("DB_PATH") ? |_| ServerErr("DB_PATH not set on environment")

    stmt =
        Sqlite.prepare!(
            {
                path: db_path,
                query: "SELECT id, task FROM todos;",
            },
        )
        ? |err| ServerErr("Failed to prepare Sqlite statement: ${Inspect.to_str(err)}")

    returned_tasks =
        Sqlite.query_many_prepared!(
            {
                stmt,
                bindings: [],
                rows: {
                    Sqlite.decode_record <-
                        id: Sqlite.i64("id"),
                        task: Sqlite.str("task"),
                },
            },
        )?
        |> List.map(|{ id, task }| "row ${Num.to_str(id)}, task: ${task}")
        |> Str.join_with("\n")

    dbg returned_tasks

    Ok({ stmt })

In the valgrind output you can see the UnhandledRows error that comes from here, I'm going to change that to a more helpful error :)

When that error happens it still should not crash with invalid pointer though, I'll investigate that next.

[Anton-4]

it still should not crash with invalid pointer though, I'll investigate that next.

status update:
I think the problem happens in this function, now I just need to be able to step through there with a debugger.

[Anton-4]

Running very similar code with just basic-cli does not result in a invalid pointer, this could be due to Tokio and the threads we use with basic-webserver.

[Anton-4]

The abort happens on call _Attr__dec_13 when trying to decrement the reference count of the first record field list_todos_stmt. I think ptr %"#arg1", the arg for #Attr_#dec_12" is incorrect.

[Anton-4]

I think the problem happens in this function,

Now I think it's in the llvm IR of Sqlite_execute_prepared:

define internal fastcc void @"Sqlite_execute_prepared!_4f1fdebc72623b7987dfbf57d7b81537b885c9e4ce317a63dbf262856adf"({ %list.RocList, ptr } %"289", ptr %0) !dbg !66 {
entry:
  %tag_alloca53 = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %struct_alloca51 = alloca { { [5 x i64], i8 } }, align 8, !dbg !67
  %load_element50 = alloca { [5 x i64], i8 }, align 8, !dbg !67
  %tag_alloca45 = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %struct_alloca43 = alloca { { [5 x i64], i8 } }, align 8, !dbg !67
  %tag_alloca40 = alloca { [5 x i64], i8 }, align 8, !dbg !67
  %tag_alloca37 = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %tag_alloca23 = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %struct_alloca21 = alloca { { [5 x i64], i8 } }, align 8, !dbg !67
  %load_element20 = alloca { [5 x i64], i8 }, align 8, !dbg !67
  %result_value9 = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %result_value6 = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %tag_alloca = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %struct_alloca = alloca { { [5 x i64], i8 } }, align 8, !dbg !67
  %load_element4 = alloca { [5 x i64], i8 }, align 8, !dbg !67
  %result_value = alloca { [6 x i64], i8 }, align 8, !dbg !67
  %struct_field_access_record_0 = extractvalue { %list.RocList, ptr } %"289", 0, !dbg !67
  %struct_field_access_record_1 = extractvalue { %list.RocList, ptr } %"289", 1, !dbg !67
  call fastcc void @"#Attr_#inc_9"(ptr %struct_field_access_record_1, i64 1), !dbg !67
  call fastcc void @"Sqlite_bind!_dc6b1b42abfea2844b7c4985e150c54b77c535b47ead6acd59d6a08b80d2"(ptr %struct_field_access_record_1, %list.RocList %struct_field_access_record_0, ptr %result_value), !dbg !67
  %tag_id_ptr = getelementptr inbounds { [6 x i64], i8 }, ptr %result_value, i32 0, i32 1, !dbg !67
  %load_tag_id = load i8, ptr %tag_id_ptr, align 1, !dbg !67
  %eq_u8 = icmp eq i8 1, %load_tag_id, !dbg !67
  br i1 %eq_u8, label %then_block, label %else_block, !dbg !67

then_block:                                       ; preds = %entry
  %get_opaque_data_ptr = getelementptr inbounds { [6 x i64], i8 }, ptr %result_value, i32 0, i32 0, !dbg !67
  %get_opaque_data_ptr1 = getelementptr inbounds { {} }, ptr %get_opaque_data_ptr, i32 0, i32 0, !dbg !67
  %load_element = load {}, ptr %get_opaque_data_ptr1, align 1, !dbg !67
  br label %joinpointcont, !dbg !67

; else_block:                                       ; preds = %entry
...

joinpointcont:                                    ; preds = %then_block
  %joinpointarg = phi {} [ %load_element, %then_block ], !dbg !67
  call fastcc void @"#Attr_#inc_9"(ptr %struct_field_access_record_1, i64 1), !dbg !67
  call fastcc void @"Sqlite_step!_594462766f77f828358545ebdadebc7d564d3daf466672cbde673babcf18c3c2"(ptr %struct_field_access_record_1, ptr %result_value6), !dbg !67
  call fastcc void @"Sqlite_reset!_d5c9642a64db206d88ab43bd2b9527b05aca746579abd7472d977da8e33ac"(ptr %struct_field_access_record_1, ptr %result_value9), !dbg !67
  %tag_id_ptr10 = getelementptr inbounds { [6 x i64], i8 }, ptr %result_value9, i32 0, i32 1, !dbg !67
  %load_tag_id11 = load i8, ptr %tag_id_ptr10, align 1, !dbg !67
  %eq_u812 = icmp eq i8 1, %load_tag_id11, !dbg !67
  br i1 %eq_u812, label %then_block13, label %else_block14, !dbg !67

then_block13:                                     ; preds = %joinpointcont
  %get_opaque_data_ptr15 = getelementptr inbounds { [6 x i64], i8 }, ptr %result_value9, i32 0, i32 0, !dbg !67
  %get_opaque_data_ptr16 = getelementptr inbounds { {} }, ptr %get_opaque_data_ptr15, i32 0, i32 0, !dbg !67
  %load_element17 = load {}, ptr %get_opaque_data_ptr16, align 1, !dbg !67
  br label %joinpointcont7, !dbg !67

; else_block14:                                     ; preds = %joinpointcont
...

joinpointcont7:                                   ; preds = %then_block13
  %joinpointarg8 = phi {} [ %load_element17, %then_block13 ], !dbg !67
  %tag_id_ptr26 = getelementptr inbounds { [6 x i64], i8 }, ptr %result_value6, i32 0, i32 1, !dbg !67
  %load_tag_id27 = load i8, ptr %tag_id_ptr26, align 1, !dbg !67
  %eq_u828 = icmp eq i8 1, %load_tag_id27, !dbg !67
  br i1 %eq_u828, label %then_block29, label %else_block30, !dbg !67

then_block29:                                     ; preds = %joinpointcont7
  %get_opaque_data_ptr31 = getelementptr inbounds { [6 x i64], i8 }, ptr %result_value6, i32 0, i32 0, !dbg !67
  %get_opaque_data_ptr32 = getelementptr inbounds { i1 }, ptr %get_opaque_data_ptr31, i32 0, i32 0, !dbg !67
  %load_element33 = load i1, ptr %get_opaque_data_ptr32, align 1, !dbg !67
  %eq_i1 = icmp eq i1 false, %load_element33, !dbg !67
  br i1 %eq_i1, label %then_block35, label %else_block36, !dbg !67

; else_block30:                                     ; preds = %joinpointcont7
...

; then_block35:                                     ; preds = %then_block29
...

else_block36:                                     ; preds = %then_block29
  %data_buffer41 = getelementptr inbounds { [5 x i64], i8 }, ptr %tag_alloca40, i32 0, i32 0, !dbg !67
  store {} zeroinitializer, ptr %data_buffer41, align 1, !dbg !67
  %tag_id_ptr42 = getelementptr inbounds { [5 x i64], i8 }, ptr %tag_alloca40, i32 0, i32 1, !dbg !67
  store i8 1, ptr %tag_id_ptr42, align 1, !dbg !67
  %struct_field_gep44 = getelementptr inbounds { { [5 x i64], i8 } }, ptr %struct_alloca43, i32 0, i32 0, !dbg !67
  call void @llvm.memcpy.p0.p0.i64(ptr align 8 %struct_field_gep44, ptr align 8 %tag_alloca40, i64 ptrtoint (ptr getelementptr ({ [5 x i64], i8 }, ptr null, i32 1) to i64), i1 false), !dbg !67
  %data_buffer46 = getelementptr inbounds { [6 x i64], i8 }, ptr %tag_alloca45, i32 0, i32 0, !dbg !67
  call void @llvm.memcpy.p0.p0.i64(ptr align 8 %data_buffer46, ptr align 8 %struct_alloca43, i64 ptrtoint (ptr getelementptr ({ { [5 x i64], i8 } }, ptr null, i32 1) to i64), i1 false), !dbg !67
  %tag_id_ptr47 = getelementptr inbounds { [6 x i64], i8 }, ptr %tag_alloca45, i32 0, i32 1, !dbg !67
  store i8 0, ptr %tag_id_ptr47, align 1, !dbg !67
  call void @llvm.memcpy.p0.p0.i64(ptr align 8 %0, ptr align 8 %tag_alloca45, i64 ptrtoint (ptr getelementptr ({ [6 x i64], i8 }, ptr null, i32 1) to i64), i1 false), !dbg !67
  ret void, !dbg !67
}

Claude is confident 😄 :

Looking at this LLVM IR, I can see a classic case of returning stack-allocated memory that becomes invalid after the function returns. This is almost certainly causing your invalid free/delete issue.
The key problem is in the final part of the function:

%data_buffer46 = getelementptr inbounds { [6 x i64], i8 }, ptr %tag_alloca45, i32 0, i32 0
call void @llvm.memcpy.p0.p0.i64(ptr align 8 %data_buffer46, ptr align 8 %struct_alloca43, i64 ..., i1 false)
%tag_id_ptr47 = getelementptr inbounds { [6 x i64], i8 }, ptr %tag_alloca45, i32 0, i32 1
store i8 0, ptr %tag_id_ptr47, align 1
call void @llvm.memcpy.p0.p0.i64(ptr align 8 %0, ptr align 8 %tag_alloca45, i64 ..., i1 false)

The issue is:

1. %tag_alloca45 is a stack allocation (visible at the top of the function with all the other alloca calls)
2. %struct_alloca43 is also stack-allocated
3. The function copies data from these stack buffers to the output parameter %0
4. When the function returns, all stack allocations are invalidated

If the structure being returned contains any pointers to these stack-allocated buffers, those pointers become dangling when the function returns. When the caller later tries to free or access the memory through these pointers, you get the invalid free error.
Based on the name "Sqlite_execute_prepared", this looks like code related to SQLite statement execution. It's likely that the returned structure contains pointers to temporary resources that are only valid during the function's execution.
To fix this, you'd need to ensure the returned data structure either:

1. Contains only values, not pointers to stack memory
2. Has proper heap-allocated memory for any data that needs to persist after the function returns
3. Uses a proper reference counting or ownership system if passing complex objects

Without seeing the calling code and data structure definitions, I can't be more specific about the exact fix, but the root cause is clearly stack memory being indirectly returned to the caller.

[Anton-4]

For completeness, the corresponding Roc code:

execute_prepared! :
    {
        stmt : Stmt,
        bindings : List Binding,
    }
    => Result {} [SqliteErr ErrCode Str, RowsReturnedUseQueryInstead]
execute_prepared! = |{ stmt, bindings }|
    try(bind!, stmt, bindings)
    res = step!(stmt)
    try(reset!, stmt)

    ##Ok({})

    when res is
        Ok(Done) ->
            Ok({})

        Ok(Row) ->
            Err(RowsReturnedUseQueryInstead)

        Err(e) ->
            Err(e)

Attached is the modified version of examples/sqlite.roc that produces the invalid pointer error:
sqlite.roc.tar.gz

[Anton-4]

Looking at the llvm IR, this error may be related to roc-lang/roc#6916

[Anton-4]

I have asked @bhansconnect for help.

[bhansconnect]

I have a rough idea what is going on, but have not thought through it fully.

The webserver is setup with sqlite statements being allocated in a special bucket. That bucket never expects any freeing. Instead, statements are created on init and then set to constant and leaked. Some reason, running a statement in init is leading to the statement being freed (not sure why, refcounting should still block this). As a results, we are trying to free a bucket allocated statement via libc free. So I see potentially two bugs here:

1. we may have a refcounting issue
2. we should allow freeing of sql statements and handle freeing from the bucket correctly.

for 2 (which may fix this bug), I think we just need some like this in roc_dealloc:

    let heap = roc_sqlite::heap();
    if heap.in_range(c_ptr) {
        heap.dealloc(c_ptr);
        return;
    }

Can someone test, I am not consistently hitting the issue right now.

[Anton-4]

Fix confirmed 🎉