Skip to content

Correct output of debug info#753

Merged
Yaxuan-w merged 1 commit intomainfrom
752-fix-bug-on-wasmtestreport-on-debug-flag
Feb 18, 2026
Merged

Correct output of debug info#753
Yaxuan-w merged 1 commit intomainfrom
752-fix-bug-on-wasmtestreport-on-debug-flag

Conversation

@m-hemmings
Copy link
Member

Motivation

Deterministic tests in scripts/wasmtestreport.py compare expected/native output against WASM output exactly.

When lind_debug is enabled, host-side runtime diagnostics such as:

  • [LIND DEBUG NUM]: 0
  • [LIND DEBUG STR]: LIND DEGUG INIT

are emitted on stderr.

Previously, successful WASM execution returned stdout + stderr, which polluted deterministic comparisons and produced false Output_mismatch failures even when the program behavior was correct.

What changed

In run_compiled_wasm():

  • If returncode == 0, the function now returns stdout only.
  • If returncode != 0, the function still returns stdout + stderr to preserve debugging context for real failures.
  • Timeout and exception handling behavior is unchanged.

Impact

  • Fixes false negatives in deterministic tests when debug logging is enabled.
  • Does not hide diagnostics for compile/runtime failures.

@m-hemmings m-hemmings linked an issue Feb 17, 2026 that may be closed by this pull request
@github-actions
Copy link
Contributor

End-to-End Test Report

Test Preview

Test Report

Deterministic Tests

Summary

MetricCount
Total Test Cases149
Number of Successes149
Number of Failures0
Number of Compilation Failure Native0
Number of Runtime Failure Native0
Number of Segmentation Fault Native0
Number of Timeout During Native0
Number of Lind Wasm Compile Failure0
Number of Lind Wasm Runtime Failure0
Number of Lind Wasm Segmentation Failure0
Number of Timeout During Lind Wasm run0
Number of Unknown Failure0
Number of C Compiler and Wasm Output mismatch0
Number of Fail Test: Native Succeeded (Should Fail)0
Number of Fail Test: Wasm Succeeded (Should Fail)0
Number of Fail Test: Both Native and Wasm Succeeded (Should Fail)0
Number of Fail Test: Native Compilation Failure (Should Succeed)0
Number of Fail Test: Wasm Compilation Failure (Should Succeed)0

Test Results by Category

Test CaseStatusError TypeNative TimeWasm TimeOutput
File Tests
chartests.cSuccessNone0.051347s4.260759s
Success
chdir_getcwd.cSuccessNone0.050128s4.230722s
Success
chmod.cSuccessNone0.055761s4.210215s
Success
clock_gettime_highlevel.cSuccessNone0.121102s4.403374s
Success
clock_gettime_simple.cSuccessNone0.044777s4.184698s
Success
cloexec.cSuccessNone0.054468s5.237627s
Success
close.cSuccessNone0.064178s4.257893s
Success
creat_access.cSuccessNone0.053682s4.206448s
Success
doubleclose.cSuccessNone0.046277s3.792817s
Success
dup.cSuccessNone0.046604s4.198536s
Success
dup2.cSuccessNone0.052718s3.848148s
Success
dup3.cSuccessNone0.052438s4.263885s
Success
dupwrite.cSuccessNone0.053679s3.806466s
Success
fchdir.cSuccessNone0.056929s4.218912s
Success
fchmod.cSuccessNone0.058633s4.195290s
Success
fcntl.cSuccessNone0.054241s4.201035s
Success
fdatasync.cSuccessNone0.051952s4.180432s
Success
filetest.cSuccessNone0.052801s3.815637s
Success
filetest1000.cSuccessNone0.061477s3.821962s
Success
flock.cSuccessNone0.061654s4.277524s
Success
fstat.cSuccessNone0.056929s4.212203s
Success
fstatfs.cSuccessNone0.047334s4.173497s
Success
fsync.cSuccessNone0.051706s4.189942s
Success
ftruncate.cSuccessNone0.058078s4.259521s
Success
getcwd.cSuccessNone0.050749s4.245371s
Success
getrandom.cSuccessNone0.053187s4.225819s
Success
ioctl.cSuccessNone0.057705s4.191678s
Success
link.cSuccessNone0.056578s4.232334s
Success
lseek.cSuccessNone0.057895s4.272264s
Success
mkdir_rmdir.cSuccessNone0.055262s4.210522s
Success
open.cSuccessNone0.047400s4.158236s
Success
pread_pwrite.cSuccessNone0.050476s4.182715s
Success
printf.cSuccessNone0.045164s3.808202s
Success
read.cSuccessNone0.053798s4.173425s
Success
readbytes.cSuccessNone0.050181s3.830541s
Success
readlink.cSuccessNone0.053951s4.199692s
Success
readlinkat.cSuccessNone0.055462s4.184459s
Success
rename.cSuccessNone0.056729s4.194025s
Success
sc-writev.cSuccessNone0.052020s4.181959s
Success
stat.cSuccessNone0.055020s4.194409s
Success
statfs.cSuccessNone0.048784s4.174783s
Success
sync_file_range.cSuccessNone0.050770s4.221106s
Success
truncate.cSuccessNone0.056918s4.249992s
Success
unlink.cSuccessNone0.055581s4.219158s
Success
unlinkat.cSuccessNone0.055120s4.230089s
Success
write.cSuccessNone0.047154s3.804788s
Success
writeloop.cSuccessNone0.057230s3.802715s
Success
writepartial.cSuccessNone0.053899s3.813142s
Success
writev.cSuccessNone0.055422s4.224170s
Success
Math Tests
math_link_smoke.cSuccessNone0.056543s3.843490s
Success
math_tests.cSuccessNone0.060291s3.998795s
Success
Memory Tests
brk.cSuccessNone0.050924s4.202455s
Success
malloc.cSuccessNone0.048900s3.802582s
Success
malloc_large.cSuccessNone0.050142s4.179936s
Success
memcpy.cSuccessNone0.048559s3.776538s
Success
memory_error_test.cSuccessNone0.055960s4.214605s
Success
mmap.cSuccessNone0.046078s4.171956s
Success
mmap_aligned.cSuccessNone0.049320s4.201695s
Success
mmap_complicated.cSuccessNone0.052874s4.245583s
Success
mmap_file.cSuccessNone0.053897s4.192592s
Success
mmap_shared.cSuccessNone0.052737s4.211867s
Success
mmaptest.cSuccessNone0.049577s4.224077s
Success
mprotect.cSuccessNone0.046928s4.179021s
Success
mprotect_boundary.cSuccessNone0.048060s4.193199s
Success
mprotect_end_region.cSuccessNone0.047196s4.172277s
Success
mprotect_middle_region.cSuccessNone0.047012s4.187056s
Success
mprotect_multiple_times.cSuccessNone0.047972s4.193797s
Success
mprotect_same_value.cSuccessNone0.047637s4.206315s
Success
mprotect_spanning_regions.cSuccessNone0.048669s4.183746s
Success
sbrk.cSuccessNone0.049153s4.208394s
Success
segfault.cSuccessNone0.056774s4.219559s
Success
shm.cSuccessNone0.054289s4.245677s
Success
shmtest.cSuccessNone0.047796s4.200931s
Success
vtable.cSuccessNone0.063328s3.850936s
Success
Networking Tests
dnstest.cSuccessNone0.054490s4.269632s
Success
epollcreate1.cSuccessNone0.058130s4.279028s
Success
gethostname.cSuccessNone0.048600s3.835842s
Success
getsockname.cSuccessNone0.056863s4.230834s
Success
getsockopt.cSuccessNone0.058387s4.245374s
Success
makepipe.cSuccessNone0.046909s3.795819s
Success
pipe.cSuccessNone0.058536s4.265664s
Success
pipe2.cSuccessNone0.055712s4.184431s
Success
pipeinput.cSuccessNone0.057024s4.261736s
Success
pipeinput2.cSuccessNone0.057682s4.261025s
Success
pipeonestring.cSuccessNone0.059653s4.282823s
Success
pipepong.cSuccessNone0.056901s4.263107s
Success
pipewrite.cSuccessNone0.052116s4.255431s
Success
poll.cSuccessNone0.055403s4.216366s
Success
recvfrom-sendto.cSuccessNone0.056242s4.219493s
Success
serverclient.cSuccessNone0.057273s3.879576s
Success
shutdown.cSuccessNone0.058825s3.881687s
Success
shutdown_fork.cSuccessNone0.056775s4.224935s
Success
simple-select.cSuccessNone0.059611s4.265161s
Success
simple_epoll.cSuccessNone0.055359s4.233703s
Success
socket.cSuccessNone0.058402s3.865850s
Success
socket_cloexec.cSuccessNone0.053280s4.217123s
Success
socketepoll.cSuccessNone0.054396s3.835789s
Success
socketpair.cSuccessNone0.054049s4.286818s
Success
socketselect.cSuccessNone0.054370s4.222615s
Success
tcp_connect_single.cSuccessNone0.056268s4.234635s
Success
uds-getsockname.cSuccessNone0.055380s4.213334s
Success
uds-socketselect.cSuccessNone0.055273s3.847802s
Success
Process Tests
chain_thread.cSuccessNone1.056495s5.365212s
Success
exit.cSuccessNone0.053378s3.787264s
Success
fork2malloc.cSuccessNone0.056147s4.233491s
Success
fork_select.cSuccessNone0.053333s4.245023s
Success
fork_simple.cSuccessNone0.053677s4.294763s
Success
fork_syscall.cSuccessNone0.059697s4.313260s
Success
forkandopen.cSuccessNone0.056701s4.276169s
Success
forkdup.cSuccessNone0.056429s3.853653s
Success
forkexecuid.cSuccessNone0.052518s5.838632s
Success
forkexecv-arg.cSuccessNone0.053847s5.630603s
Success
forkexecv.cSuccessNone0.052728s5.718756s
Success
forkfiles.cSuccessNone0.056325s4.239097s
Success
forkmalloc.cSuccessNone0.068750s4.305446s
Success
forknodup.cSuccessNone0.056609s4.225991s
Success
function-ptr.cSuccessNone0.052394s4.271128s
Success
getegid_syscall.cSuccessNone0.055875s4.262774s
Success
getgid_syscall.cSuccessNone0.055967s4.263881s
Success
getpid.cSuccessNone0.047526s4.196568s
Success
getpid_syscall.cSuccessNone0.056787s4.277085s
Success
getppid.cSuccessNone0.056470s4.305372s
Success
getppid_syscall.cSuccessNone0.058453s4.238013s
Success
getuid.cSuccessNone0.056858s4.242620s
Success
getuid_syscall.cSuccessNone0.053807s4.247841s
Success
hello-arg.cSuccessNone0.044731s3.825070s
Success
hello.cSuccessNone0.045324s3.815173s
Success
longjmp.cSuccessNone0.046093s3.832554s
Success
mutex.cSuccessNone2.061641s6.539640s
Success
sem_forks.cSuccessNone0.059021s4.248331s
Success
setsid.cSuccessNone0.047374s3.816643s
Success
template.cSuccessNone0.054702s4.249344s
Success
test_exec_nofork.cSuccessNone0.053282s5.799000s
Success
test_unlink_open_file.cSuccessNone0.049047s3.799556s
Success
thread.cSuccessNone0.050014s4.346827s
Success
uname.cSuccessNone0.049219s4.273065s
Success
wait.cSuccessNone2.052683s5.847322s
Success
waitpid_anychild.cSuccessNone0.056248s4.240694s
Success
waitpid_syscall.cSuccessNone1.055963s5.226780s
Success
waitpid_wnohang.cSuccessNone0.055579s3.833202s
Success
Signal Tests
alarm.cSuccessNone7.055599s10.867727s
Success
setitimer.cSuccessNone7.054715s10.841527s
Success
sigalrm.cSuccessNone2.054946s5.836210s
Success
sigchld.cSuccessNone1.056396s4.873618s
Success
signal-fork.cSuccessNone4.057463s7.883913s
Success
signal-simple.cSuccessNone0.054885s3.827275s
Success
signal_fork.cSuccessNone0.052700s4.244526s
Success
signal_int_ignored.cSuccessNone2.055328s6.270663s
Success
sigprocmask.cSuccessNone1.053834s4.841177s
Success
Fail Tests

Summary

MetricCount
Total Test Cases2
Number of Successes2
Number of Failures0
Number of Compilation Failure Native0
Number of Runtime Failure Native0
Number of Segmentation Fault Native0
Number of Timeout During Native0
Number of Lind Wasm Compile Failure0
Number of Lind Wasm Runtime Failure0
Number of Lind Wasm Segmentation Failure0
Number of Timeout During Lind Wasm run0
Number of Unknown Failure0
Number of C Compiler and Wasm Output mismatch0
Number of Fail Test: Native Succeeded (Should Fail)0
Number of Fail Test: Wasm Succeeded (Should Fail)0
Number of Fail Test: Both Native and Wasm Succeeded (Should Fail)0
Number of Fail Test: Native Compilation Failure (Should Succeed)0
Number of Fail Test: Wasm Compilation Failure (Should Succeed)0

Test Results by Category

Test CaseStatusError TypeNative TimeWasm TimeOutput
Memory Tests
mmap-negative1.cSuccessNone0.119847s4.503752s
Success
mmap-negative2.cSuccessNone0.117563s4.060055s
Success

@Yaxuan-w Yaxuan-w merged commit f815ba0 into main Feb 18, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix bug on wasmtestreport on debug flag

3 participants