Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlite3_file_control() support #1000

Merged
merged 1 commit into from
Jan 28, 2022

Conversation

benbjohnson
Copy link
Contributor

@benbjohnson benbjohnson commented Jan 16, 2022

This pull request adds the SQLiteConn.FileControl() method which calls the underlying sqlite3_file_control() function. This can be used for low-level operations on SQLite databases such as persisting the WAL file after database close.

Fixes #999

sqlite3.go Outdated
// return code are both opcode-specific. Please see the SQLite documentation.
//
// See: sqlite3_file_control, https://www.sqlite.org/c3ref/file_control.html
func (c *SQLiteConn) FileControl(dbName string, op int, arg unsafe.Pointer) int {
Copy link
Collaborator

@rittneje rittneje Jan 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last argument seems somewhat troubling to me. For example, for some of the ops, it needs to be a pointer to some C struct, but the caller will be hard-pressed to do that within the confines of Go. Consequently, I'm not sure we really want to expose sqlite3_file_control directly like this.

Taking a page from the syscall package, we could do something like SetFileControlInt/GetFileControlInt, and use a more user-friendly type for the final parameter, performing the conversion to pointer internally.

Alternatively, we can make a method specifically for getting/setting the persistent WAL mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I switched it to FileControlInt() instead. 👍

sqlite3_test.go Outdated
ConnectHook: func(conn *SQLiteConn) error {
arg := int(1)
if rc := conn.FileControl("", SQLITE_FCNTL_PERSIST_WAL, unsafe.Pointer(&arg)); rc != 0 {
t.Fatalf("Unexpected return code from FileControl(): %d", rc)
Copy link
Collaborator

@rittneje rittneje Jan 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably should not call t.Fatalf here, since this may be happening on some other goroutine, depending how database/sql works. Instead, just return an error. Below, call db.Ping after sql.Open and move the assertion there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. It now returns an error instead.

@benbjohnson
Copy link
Contributor Author

@rittneje I fixed up the PR. Can you take another look please?

sqlite3.go Outdated
// Standard File Control Opcodes
// See: https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html
const (
SQLITE_FCNTL_LOCKSTATE = 1
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQLITE_FCNTL_LOCKSTATE = int(C.SQLITE_FCNTL_LOCKSTATE)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed 👍

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use C.SQLITE_FCNTL_XXX instead of number literals?

Copy link
Contributor Author

@benbjohnson benbjohnson Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattn Good call. I updated it and push up a new commit. 👍

sqlite3.go Outdated Show resolved Hide resolved
@benbjohnson
Copy link
Contributor Author

@rittneje @mattn Are there any other changes I should make to get this merged? Thanks!

@codecov-commenter
Copy link

codecov-commenter commented Jan 27, 2022

Codecov Report

Merging #1000 (c640ad0) into master (671e666) will increase coverage by 0.05%.
The diff coverage is 77.77%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1000      +/-   ##
==========================================
+ Coverage   45.98%   46.04%   +0.05%     
==========================================
  Files          11       11              
  Lines        1481     1490       +9     
==========================================
+ Hits          681      686       +5     
- Misses        665      667       +2     
- Partials      135      137       +2     
Impacted Files Coverage Δ
sqlite3.go 52.97% <77.77%> (+0.02%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 671e666...c640ad0. Read the comment docs.

@benbjohnson
Copy link
Contributor Author

benbjohnson commented Jan 27, 2022

@rittneje @mattn I removed the 4 SQLITE_FCNTL_ that were failing in CI build. Can you rerun CI please?

@mattn
Copy link
Owner

mattn commented Jan 27, 2022

@rittneje good to merge?

@@ -297,6 +297,47 @@ const (
/*SQLITE_RECURSIVE = C.SQLITE_RECURSIVE*/
)

// Standard File Control Opcodes
// See: https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html
const (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattn By referencing the actual C constants here, this won't compile for anyone linking with an older version of SQLite (via the libsqlite3 tag). For example, SQLITE_FCNTL_SIZE_LIMIT wasn't added until 3.27.0.

How do you want to handle this?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't consider it. @benbjohnson Sorry much. Could you please rollback changes for the constants?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattn @rittneje No problem. I reverted the code back to using hardcoded integers. 👍

This commit adds the SQLiteConn.FileControlInt() method which calls the
underlying sqlite3_file_control() function with an int argument. This can
be used for low-level operations on SQLite databases such as persisting
the WAL file after database close.
Copy link
Collaborator

@rittneje rittneje left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mattn mattn merged commit ae2a61f into mattn:master Jan 28, 2022
@mattn
Copy link
Owner

mattn commented Jan 28, 2022

Thank you

@benbjohnson
Copy link
Contributor Author

@mattn @rittneje Thanks so much for all the reviews! 🙏

@benbjohnson benbjohnson deleted the sqlite3-file-control branch January 28, 2022 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support sqlite3_file_control()
4 participants