Skip to content

tests: refactor grate unit tests to be self-validating#745

Merged
Yaxuan-w merged 1 commit intomainfrom
grate-self-validating
Feb 17, 2026
Merged

tests: refactor grate unit tests to be self-validating#745
Yaxuan-w merged 1 commit intomainfrom
grate-self-validating

Conversation

@celinehoang177
Copy link
Contributor

Closes #729

  • Cage files now validate return values internally and exit(EXIT_FAILURE) on mismatch
  • Grate files now check child exit status and propagate failures
  • Fixed --compile-grate flag in lind_compile
  • Naming convention enforced: cage .c, grate _grate.c

@github-actions
Copy link
Contributor

End-to-End Test Report

Test Preview

Test Report

Deterministic Tests

Summary

MetricCount
Total Test Cases148
Number of Successes147
Number of Failures1
Number of Compilation Failure Native0
Number of Runtime Failure Native0
Number of Segmentation Fault Native0
Number of Timeout During Native0
Number of Lind Wasm Compile Failure1
Number of Lind Wasm Runtime Failure0
Number of Lind Wasm Segmentation Failure0
Number of Timeout During Lind Wasm run0
Number of Unknown Failure0
Number of C Compiler and Wasm Output mismatch0
Number of Fail Test: Native Succeeded (Should Fail)0
Number of Fail Test: Wasm Succeeded (Should Fail)0
Number of Fail Test: Both Native and Wasm Succeeded (Should Fail)0
Number of Fail Test: Native Compilation Failure (Should Succeed)0
Number of Fail Test: Wasm Compilation Failure (Should Succeed)0

Test Results by Category

Test CaseStatusError TypeNative TimeWasm TimeOutput
File Tests
chartests.cSuccessNone0.050088s4.554478s
Success
chdir_getcwd.cSuccessNone0.051335s4.558907s
Success
chmod.cSuccessNone0.057381s4.565720s
Success
clock_gettime_highlevel.cSuccessNone0.122857s4.756925s
Success
clock_gettime_simple.cSuccessNone0.044775s4.497683s
Success
cloexec.cSuccessNone0.055115s6.180762s
Success
close.cSuccessNone0.063573s4.580309s
Success
creat_access.cSuccessNone0.054385s4.550058s
Success
doubleclose.cSuccessNone0.045764s4.517028s
Success
dup.cSuccessNone0.045607s4.528842s
Success
dup2.cSuccessNone0.055669s4.547930s
Success
dup3.cSuccessNone0.050357s4.567556s
Success
dupwrite.cSuccessNone0.053691s4.497408s
Success
fchdir.cSuccessNone0.058714s4.569117s
Success
fchmod.cSuccessNone0.059990s4.571293s
Success
fcntl.cSuccessNone0.052220s4.551687s
Success
fdatasync.cSuccessNone0.052460s4.568724s
Success
filetest.cSuccessNone0.054109s4.554694s
Success
filetest1000.cSuccessNone0.063485s4.600436s
Success
flock.cSuccessNone0.060861s4.602878s
Success
fstat.cSuccessNone0.055914s4.555003s
Success
fstatfs.cSuccessNone0.048520s4.543324s
Success
fsync.cSuccessNone0.052547s4.543513s
Success
ftruncate.cSuccessNone0.058792s4.604493s
Success
getcwd.cSuccessNone0.049910s4.550528s
Success
getrandom.cSuccessNone0.052845s4.555487s
Success
ioctl.cSuccessNone0.060573s4.525697s
Success
link.cSuccessNone0.058107s4.592364s
Success
lseek.cSuccessNone0.057811s4.619756s
Success
mkdir_rmdir.cSuccessNone0.055303s4.557109s
Success
open.cSuccessNone0.050736s4.544349s
Success
pread_pwrite.cSuccessNone0.052441s4.542588s
Success
printf.cSuccessNone0.044801s4.502501s
Success
read.cSuccessNone0.053653s4.545638s
Success
readbytes.cSuccessNone0.049957s4.504744s
Success
readlink.cSuccessNone0.053997s4.556566s
Success
readlinkat.cSuccessNone0.060169s4.545960s
Success
rename.cSuccessNone0.056888s4.549758s
Success
sc-writev.cSuccessNone0.052899s4.602185s
Success
stat.cSuccessNone0.054562s4.572269s
Success
statfs.cSuccessNone0.050370s4.544323s
Success
sync_file_range.cSuccessNone0.052254s4.551163s
Success
truncate.cSuccessNone0.055390s4.552180s
Success
unlink.cSuccessNone0.056686s4.596583s
Success
unlinkat.cSuccessNone0.056086s4.574073s
Success
write.cSuccessNone0.047071s4.512687s
Success
writeloop.cSuccessNone0.056026s4.530589s
Success
writepartial.cSuccessNone0.055096s4.477991s
Success
writev.cSuccessNone0.055198s4.570600s
Success
Math Tests
math_link_smoke.cFailureLind_wasm_compiling0.058769s0.111065s
wasm-ld: error: unable to find library -lm
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Memory Tests
brk.cSuccessNone0.052611s4.544821s
Success
malloc.cSuccessNone0.050101s4.506186s
Success
malloc_large.cSuccessNone0.049756s4.514973s
Success
memcpy.cSuccessNone0.050401s4.492243s
Success
memory_error_test.cSuccessNone0.056231s4.528500s
Success
mmap.cSuccessNone0.045708s4.517532s
Success
mmap_aligned.cSuccessNone0.049366s4.517391s
Success
mmap_complicated.cSuccessNone0.052502s4.539746s
Success
mmap_file.cSuccessNone0.054289s4.587771s
Success
mmap_shared.cSuccessNone0.052437s4.555743s
Success
mmaptest.cSuccessNone0.048808s4.518016s
Success
mprotect.cSuccessNone0.047130s4.527288s
Success
mprotect_boundary.cSuccessNone0.048642s4.539640s
Success
mprotect_end_region.cSuccessNone0.046886s4.518269s
Success
mprotect_middle_region.cSuccessNone0.046816s4.524986s
Success
mprotect_multiple_times.cSuccessNone0.047535s4.539059s
Success
mprotect_same_value.cSuccessNone0.046764s4.526739s
Success
mprotect_spanning_regions.cSuccessNone0.047998s4.544060s
Success
sbrk.cSuccessNone0.049846s4.537273s
Success
segfault.cSuccessNone0.055613s4.563620s
Success
shm.cSuccessNone0.055184s4.571488s
Success
shmtest.cSuccessNone0.048739s4.537800s
Success
vtable.cSuccessNone0.061589s4.568598s
Success
Networking Tests
dnstest.cSuccessNone0.054140s4.587169s
Success
epollcreate1.cSuccessNone0.055230s4.576744s
Success
gethostname.cSuccessNone0.048686s4.507768s
Success
getsockname.cSuccessNone0.055902s4.554348s
Success
getsockopt.cSuccessNone0.059451s4.616468s
Success
makepipe.cSuccessNone0.046994s4.497205s
Success
pipe.cSuccessNone0.056482s4.619928s
Success
pipe2.cSuccessNone0.057681s4.519908s
Success
pipeinput.cSuccessNone0.058901s4.564348s
Success
pipeinput2.cSuccessNone0.057885s4.565983s
Success
pipeonestring.cSuccessNone0.059643s4.582774s
Success
pipepong.cSuccessNone0.058178s4.601304s
Success
pipewrite.cSuccessNone0.050732s4.529524s
Success
poll.cSuccessNone0.055858s4.570981s
Success
recvfrom-sendto.cSuccessNone0.058271s4.556939s
Success
serverclient.cSuccessNone0.055557s4.530998s
Success
shutdown.cSuccessNone0.057865s4.575130s
Success
shutdown_fork.cSuccessNone0.057406s4.608436s
Success
simple-select.cSuccessNone0.058027s4.581171s
Success
simple_epoll.cSuccessNone0.055152s4.562920s
Success
socket.cSuccessNone0.053794s4.541846s
Success
socket_cloexec.cSuccessNone0.054229s4.531199s
Success
socketepoll.cSuccessNone0.054633s4.549872s
Success
socketpair.cSuccessNone0.054436s4.604807s
Success
socketselect.cSuccessNone0.054316s4.534963s
Success
tcp_connect_single.cSuccessNone0.055204s4.567339s
Success
uds-getsockname.cSuccessNone0.054784s4.546918s
Success
uds-socketselect.cSuccessNone0.055777s4.552432s
Success
Process Tests
chain_thread.cSuccessNone1.056689s5.730985s
Success
exit.cSuccessNone0.052778s4.538813s
Success
fork2malloc.cSuccessNone0.058372s4.554603s
Success
fork_select.cSuccessNone0.055033s4.617103s
Success
fork_simple.cSuccessNone0.052472s4.551217s
Success
fork_syscall.cSuccessNone0.058718s4.621252s
Success
forkandopen.cSuccessNone0.055982s4.583708s
Success
forkdup.cSuccessNone0.056074s4.549882s
Success
forkexecuid.cSuccessNone0.054267s6.266937s
Success
forkexecv-arg.cSuccessNone0.053194s6.230837s
Success
forkexecv.cSuccessNone0.051079s6.210396s
Success
forkfiles.cSuccessNone0.057723s4.574092s
Success
forkmalloc.cSuccessNone0.070372s4.568892s
Success
forknodup.cSuccessNone0.056840s4.536237s
Success
function-ptr.cSuccessNone0.052535s4.537755s
Success
getegid_syscall.cSuccessNone0.055799s4.595990s
Success
getgid_syscall.cSuccessNone0.055610s4.610519s
Success
getpid.cSuccessNone0.047678s4.540026s
Success
getpid_syscall.cSuccessNone0.056902s4.615666s
Success
getppid.cSuccessNone0.055483s4.600957s
Success
getppid_syscall.cSuccessNone0.057841s4.589811s
Success
getuid.cSuccessNone0.055397s4.537881s
Success
getuid_syscall.cSuccessNone0.053929s4.580533s
Success
hello-arg.cSuccessNone0.045169s4.497178s
Success
hello.cSuccessNone0.045188s4.564983s
Success
longjmp.cSuccessNone0.047391s4.543268s
Success
mutex.cSuccessNone2.061701s6.832773s
Success
sem_forks.cSuccessNone0.058697s4.582665s
Success
setsid.cSuccessNone0.048626s4.502328s
Success
template.cSuccessNone0.054717s4.573417s
Success
test_exec_nofork.cSuccessNone0.053120s6.231790s
Success
test_unlink_open_file.cSuccessNone0.049450s4.535683s
Success
thread.cSuccessNone0.049360s4.700771s
Success
uname.cSuccessNone0.048165s4.533503s
Success
wait.cSuccessNone2.053178s6.552018s
Success
waitpid_anychild.cSuccessNone0.054464s4.564075s
Success
waitpid_syscall.cSuccessNone1.055862s5.590778s
Success
waitpid_wnohang.cSuccessNone0.055078s4.593576s
Success
Signal Tests
alarm.cSuccessNone7.056366s11.543373s
Success
setitimer.cSuccessNone7.053845s11.544652s
Success
sigalrm.cSuccessNone2.054793s6.540310s
Success
sigchld.cSuccessNone1.055679s5.556341s
Success
signal-fork.cSuccessNone4.058893s8.574247s
Success
signal-simple.cSuccessNone0.054912s4.524515s
Success
signal_fork.cSuccessNone0.050778s4.565401s
Success
signal_int_ignored.cSuccessNone2.053832s6.587115s
Success
sigprocmask.cSuccessNone1.055336s5.566951s
Success
Fail Tests

Summary

MetricCount
Total Test Cases2
Number of Successes2
Number of Failures0
Number of Compilation Failure Native0
Number of Runtime Failure Native0
Number of Segmentation Fault Native0
Number of Timeout During Native0
Number of Lind Wasm Compile Failure0
Number of Lind Wasm Runtime Failure0
Number of Lind Wasm Segmentation Failure0
Number of Timeout During Lind Wasm run0
Number of Unknown Failure0
Number of C Compiler and Wasm Output mismatch0
Number of Fail Test: Native Succeeded (Should Fail)0
Number of Fail Test: Wasm Succeeded (Should Fail)0
Number of Fail Test: Both Native and Wasm Succeeded (Should Fail)0
Number of Fail Test: Native Compilation Failure (Should Succeed)0
Number of Fail Test: Wasm Compilation Failure (Should Succeed)0

Test Results by Category

Test CaseStatusError TypeNative TimeWasm TimeOutput
Memory Tests
mmap-negative1.cSuccessNone0.113827s5.097368s
Success
mmap-negative2.cSuccessNone0.129597s4.813956s
Success

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does previous version not work?

@Yaxuan-w Yaxuan-w merged commit 22d59ed into main Feb 17, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: refactor grate unit tests to be self-validating

2 participants