sync cr-sqlite to ext/crr

[tantaman]

#147

copy cr-sqlite extension code as-is to ext/crr

build & test

https://app.warp.dev/block/FWDFdN9HSg4jaHayk3EwBo
https://app.warp.dev/block/lseqHcw4cpmyqWMBsyexs5

Next steps:

• remove src/sqlite/shell.c,sqlite3.c,sqlite3.h,sqlite3ext.h and use the files present in libsql
• Integrate w/ autotools
• integrate tests with your CI
• remove reliance on commit_hook
• draft new create crr syntax
• fuzzing?

Note: the package.json and other js files are for distribution of this extension on npm.

There are also a number of integration tests in Python and TypeScript (using https://github.com/dubzzz/fast-check) not included here. I'm thinking of the best way to include those.

[psarna]

thanks @tantaman! How can I verify that everything works, manually? I tried going to crr/rs/core and running cargo build from there, but it fails to find a deps/sqlite3ext.h when running sqlite-rs-embedded/sqlite3_capi/build.rs. I saw that the original repo has a Makefile one can use to build the extension, but it's not part of this PR

[tantaman]

hmm. Let me double check that I got everything.

[tantaman]

make loadable in the crr directory should do the trick.

(btw -- you need to be on the Rust nightly toolchain. Let me add an item to add +nightly to the makefile so you don't need to change your default toolchain)

> pwd
libsql/ext/crr
> make loadable
> ls dist
crsqlite.dylib			sqlite3-extra.c			test.dSYM
libcrsql_bundle-loadable.a	test

make test is also a target although it doesn't run the Rust tests at the moment.

Rust tests are in ext/crr/rs/integration-check

cargo test

[psarna]

@tantaman maybe you have Makefile in your local dir, but it wasn't checked to git and pushed? It's not present, and not listed here: https://github.com/libsql/libsql/pull/151/files

[tantaman]

@psarna - Hah, yeah. The .gitignore file in the libsql root excludes Makefile

https://github.com/libsql/libsql/blob/7ecf22a1a61e13655427b576c3a2a7c9f1ef485b/.gitignore#L3-L5

> git check-ignore crr/Makefile -v
.gitignore:4:Makefile	crr/Makefile

[tantaman]

this import of cr-sql also depends on a bundled sqlite3.c amalgamation which is rightfully ignored by libsql.

I'll need to update my make target to depend on make sqlite3.c from libsql

Will let you know when to check back.

[psarna]

Ah, right. Feel free to also amend the .gitignore file so it explicitly ignores all the Makefiles that make sense, while preserving e.g. ones from ext/

[tantaman]

@psarna - Looks good now with the exception that libsql generates a bunch of warnings that base sqlite does not.

See linked terminal output in the build & test repro (from a fresh clone this time) below.

> git clone git@github.com:vlcn-io/libsql.git libsql-testit
> cd libsql-testit/
> git checkout crsqlite
> ./configure
> cd ext/crr
> make loadable
✅ https://app.warp.dev/block/kv1cWBgm8AkKrtbXvhaa0c [1]
> make test
✅ https://app.warp.dev/block/vzmVp7NObqPZXS3wo3P7Gz [2]
> cd rs/integration-check/
> cargo test
✅ https://app.warp.dev/block/FaiyL71oy9RKRhIQqdlVNT [3]

[1] https://app.warp.dev/block/kv1cWBgm8AkKrtbXvhaa0c
[2] https://app.warp.dev/block/vzmVp7NObqPZXS3wo3P7Gz
[3] https://app.warp.dev/block/FaiyL71oy9RKRhIQqdlVNT

[psarna]

thanks, I'll go over the warnings and fix, and meanwhile let's just wait for CI to finish and we can merge the initial PR!

[psarna]

@tantaman which flags did you use to ./configure libsql before running make loadable? I'm trying to reproduce the redefinition warnings from your first warp link but they didn't happen in any configurations I tried