Skip to content

Commit

Permalink
removed wal option
Browse files Browse the repository at this point in the history
  • Loading branch information
Joshua Wise committed Sep 9, 2016
1 parent 1ad0d1a commit abba64f
Show file tree
Hide file tree
Showing 5 changed files with 47 additions and 47 deletions.
51 changes: 40 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,17 +44,7 @@ If the database is closed, the `close` event will be fired. If the database was

### Options

#### *options.memory*

If this option is `true`, an in-memory database will be created, rather than a disk-bound one. Default is `false`.

#### *options.wal*

If this option is `true` (the default), the following PRAGMA are applied:
- `PRAGMA journal_mode = WAL;`
- `PRAGMA synchronous = 1;`

This means the database will be opened in [Write Ahead Logging](https://www.sqlite.org/wal.html) mode. If you set `options.wal` to `false`, the old [Rollback Journal](https://www.sqlite.org/lockingv3.html#rollback) mode will be used, as well as the default `synchronous` setting.
If `options.memory` is `true`, an in-memory database will be created, rather than a disk-bound one. Default is `false`.

### .statement(string) -> Statement

Expand All @@ -77,6 +67,14 @@ var cacheSize = db.pragma('cache_size', true); // returns the string "32000"

The data returned by `.pragma()` is always in string format. The documentation on SQLite3 PRAGMA statements can be found [here](https://www.sqlite.org/pragma.html).

### .checkpoint([force], callback) -> this

This method is provided because [.pragma()](#pragmastring-simplify---results)'s synchronous nature makes it unsuitable for running [WAL mode checkpoints](https://www.sqlite.org/wal.html).

By default, this method will execute a checkpoint in "PASSIVE" mode, which means it might not perform a *complete* checkpoint if there are pending reads or write on the database. If the first argument is `true`, it will execute the checkpoint in "RESTART" mode, which ensures a complete checkpoint operation.

When the operation is complete, the callback is invoked with an `Error` or `null` as its first parameter, depending on if the operation was successful. If successful, the callback's second parameter will be a number between `0` and `1`, indicating the fraction of the WAL file that was checkpointed. For forceful checkpoints ("RESTART" mode), this number will always be `1`.

#### WARNING: You should NOT use prepared [statements](#statementstring---statement) or [transactions](#transactionarrayofstrings---transaction) to run PRAGMA statements. Doing so could result in database corruption.

### .close() -> this
Expand Down Expand Up @@ -189,6 +187,7 @@ Same as [`Statement#busy`](#get-busy---boolean).

Returns a concatenation of every source string that was used to create the prepared transaction. The source strings are seperated by newline characters (`\n`).


# Binding Parameters

This section applies to anywhere in the documentation that specifies the optional argument `[...bindParameters]`.
Expand Down Expand Up @@ -226,6 +225,36 @@ var stmt = db.statement('INSERT INTO people VALUES (@name, @name, ?)');
stmt.run(45, {name: 'Henry'}, callback);
```

# Performance

By default, SQLite3 databases are not well suited for some write-heavy applications. If your application reads frequently, but writes to the database very infrequently, you'll probably be fine. But if this is not the case, it's recommended to turn on [WAL mode](https://www.sqlite.org/wal.html):

```js
db.pragma('journal_mode = WAL');
```

WAL mode has a few *disadvantages* to consider:
- Transactions that involve ATTACHed databases are atomic for each individual database, but are not atomic across all databases as a set.
- Under rare circumstances, the [WAL file](https://www.sqlite.org/wal.html) may experience "checkpoint starvation" (see below).
- Some hardware/system limitations that may affect some users, [listed here](https://www.sqlite.org/wal.html).

However, you trade those disadvantages for greatly improved performance in most web applications.

If you want to *massively* improve write performance and you're willing to sacrifice a tiny bit of [durability](https://en.wikipedia.org/wiki/Durability_(database_systems)), you can use this:

```js
db.pragma('journal_mode = WAL');
db.pragma('synchronous = 1');
```

Normally, setting `synchronous = 1` would introduce the risk of database corruption following a power loss or hard reboot. But in [WAL mode](https://www.sqlite.org/wal.html), you do not introduce this risk.

### Defending against "checkpoint starvation"

Checkpoint starvation is when SQLite3 is unable to recycle the [WAL file](https://www.sqlite.org/wal.html) due to everlasting concurrent reads to the database. If this happens, the WAL file will grow without bound, leading to unacceptable amounts of disk usage and deteriorating performance.

To prevent this, you can optionally use the [db.checkpoint()](#checkpointforce-callback---this) method to force checkpointing whenever you deem appropriate.

# SQLite3 compilation options

The following [compilation options](https://www.sqlite.org/compile.html) are used:
Expand Down
5 changes: 1 addition & 4 deletions lib/database.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ function Database(filename, options) {
if (typeof options !== 'object' || options === null) {
options = {};
}
if (options.wal == null) {
options.wal = true;
}

if (!filename) {
throw new TypeError('A database filename cannot be an empty string.');
Expand All @@ -38,7 +35,7 @@ function Database(filename, options) {
+ '?mode=memory';
}

return new CPPDatabase(filename, !!options.wal, NullFactory);
return new CPPDatabase(filename, NullFactory);
}
CPPDatabase.prototype.constructor = Database;
Database.prototype = Object.create(Object.prototype, toDescriptor(CPPDatabase.prototype));
Expand Down
5 changes: 2 additions & 3 deletions src/objects/database/new.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@

NAN_METHOD(Database::New) {
REQUIRE_ARGUMENT_STRING(0, filename);
REQUIRE_ARGUMENT_BOOLEAN(1, wal);
REQUIRE_ARGUMENT_FUNCTION(2, nullFactory);
REQUIRE_ARGUMENT_FUNCTION(1, nullFactory);
Database* db = new Database();
db->Wrap(info.This());
info.This()->SetHiddenValue(NEW_INTERNAL_STRING_FAST("NullFactory"), nullFactory);

db->Ref();
db->workers += 1;
Nan::AsyncQueueWorker(new OpenWorker(db, C_STRING(filename), wal));
Nan::AsyncQueueWorker(new OpenWorker(db, C_STRING(filename)));

info.GetReturnValue().Set(info.This());
}
32 changes: 4 additions & 28 deletions src/workers/database-workers/open.cc
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,9 @@
const int WRITE_MODE = SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_SHAREDCACHE;
const int READ_MODE = SQLITE_OPEN_READONLY | SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_SHAREDCACHE;

OpenWorker::OpenWorker(Database* db, char* filename, bool wal) : Nan::AsyncWorker(NULL),
OpenWorker::OpenWorker(Database* db, char* filename) : Nan::AsyncWorker(NULL),
db(db),
filename(filename),
wal(wal) {}
filename(filename) {}
OpenWorker::~OpenWorker() {
delete[] filename;
}
Expand All @@ -32,31 +31,8 @@ void OpenWorker::Execute() {
return;
}

sqlite3_busy_timeout(db->write_handle, 30000);
sqlite3_busy_timeout(db->read_handle, 30000);

if (wal) {

char* err;
status = sqlite3_exec(db->write_handle, "PRAGMA journal_mode = WAL; PRAGMA synchronous = 1;", NULL, 0, &err);
if (status != SQLITE_OK) {
SetErrorMessage(err);
db->CloseHandles();
sqlite3_free(err);
return;
}

sqlite3_free(err);
status = sqlite3_exec(db->read_handle, "PRAGMA journal_mode = WAL; PRAGMA synchronous = 1;", NULL, 0, &err);
if (status != SQLITE_OK) {
SetErrorMessage(err);
db->CloseHandles();
sqlite3_free(err);
return;
}

sqlite3_free(err);
}
sqlite3_busy_timeout(db->write_handle, 5000);
sqlite3_busy_timeout(db->read_handle, 5000);

db->t_handles = new TransactionHandles(db->write_handle, &status);
if (status != SQLITE_OK) {
Expand Down
1 change: 0 additions & 1 deletion src/workers/database-workers/open.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ class OpenWorker : public Nan::AsyncWorker {
private:
Database* const db;
const char* const filename;
bool const wal;
};

#endif

0 comments on commit abba64f

Please sign in to comment.