v2.4.0-srh-extra

This is a back-ported release of a soft durability flush interval feature.

Funding for this was provided by ZeroTier, a company that makes network virtualization software that allows networks to be created and managed effortlessly over both LAN and WAN.

This release is called v2.4.0-srh-extra. Its cluster protocol is not compatible with the official 2.4.0 binaries. The file format is, however, downgrade compatible. You can switch to this new version, try it out, and then switch back if you don't like it.

You may also upgrade from v2.3.6-srh-extra or v2.3.5-srh-extra and keep your settings.

What's new? The same as in the previous v2.3.6-srh-extra release, except for 2.4.0, plus a bugfix:

• Everything in the official 2.4.0 release is here
• Soft durability writes now get their data flushed every 5 seconds, instead of being flushed immediately. If you write and rewrite the same documents over and over again, you might see your I/O bandwidth decrease by a factor of 1000. Or more! (Or less.)
• A new field in the table configuration called 'user_value' has been added. It's initially set to the empty object, { }. You can do the following things with it:
  1. Set it to { "srh/flush_interval": 23.3 }. This will change the flush interval to 23.3 seconds, instead of 5 seconds.
  2. Set it to { "srh/flush_interval": 0.01 }, if you want your flushes to happen relatively quickly.
  3. Set it to { "srh/flush_interval": "default" }. This sets the flush interval to the default value, 5 seconds.
  4. Set it to { "srh/flush_interval": "never" }. This makes flushing "never" happen, under ideal conditions.
  5. Set it to whatever you like, as long as it's an object. Add new fields to the object for your own operational needs.
• The bugfix: In v2.3.6-srh-extra, the server would crash (and then crash on startup!) if you set 'user_value' to something other than an object. In this release, the server both (a) recovers if 'user_value' got set to something other than an object (which might permit you to upgrade from v2.3.6-srh-extra after putting that into a broken state), and (b) prohibits you from setting the field to anything but an object. The rationale is paternalistic: You don't want to walk yourself into a corner by setting 'user_value' to something you can't add another field to.
• Another change: the fix for rethinkdb#6819 is included.

Under cache memory pressure, a flush could still happen sooner than the flush interval demands (even if set to "never"). Setting "never", or a very long flush interval, is only a good idea for tables that comfortably fit into memory. Otherwise, those tables will tend to hog memory that could be put to better use.

Example queries (in JavaScript) for configuring the flush interval of a table:

r.table('foo').config().update({'user_value': {'srh/flush_interval': 23.3}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 0.01}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 'default'}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 'never'}})

An example that sets the flush interval to 23.3 seconds, with some other user data:

r.table('foo').config().update({
    'user_value': {'srh/flush_interval': 23.3, 'john/blah': "john's data"}
})

It's not possible to set the user value to whatever you want:
r.table('foo').config().update({'user_value': "hello world"}) /* error message */

To set the user value back to what it was initially:
r.table('foo').config().update({'user_value': { }})

You might also want to set the default durability to "soft":

r.table('foo').config().update({
    'durability': 'soft',
    'user_value': {'srh/flush_interval': 'never'}
})

Warning: If you set the user value, then downgrade to RethinkDB 2.4.0, and then upgrade back to 2.4.0-srh-extra, you might lose your user value configuration. (By design, you will lose your user value configuration if you modify the table's configuration at all while running RethinkDB 2.4.0.)

You might also notice some performance improvements in secondary index creation, bringing up new replicas, and running unit tests.

You might also notice performance regressions. Please report them to https://github.com/srh/rethinkdb/issues. My take is, if you're running the official RethinkDB 2.4.0, this release is far more likely to improve performance than hurt it, so it's worth using 2.4.0-srh-extra.

Some users (running certain workloads) of v2.3.6-srh-extra reported severe memory leaks. Note that users have also reported memory leaks with v2.3.6 proper. That might be exacerbated by this release somehow. Or it might be fixed in v2.4.0. Time will tell.

Binaries are currently available for Ubuntu 19.10/18.04/16.04/14.04, Debian Buster/Stretch/Jessie, and CentOS 6/7/8. Let me know if you'd like your platform to be added. And let me know if you even noticed this release and use it, or think you might be interested in using it.

Note: The CentOS 8 and Stretch builds are based off the tag https://github.com/srh/rethinkdb/releases/v2.4.0-srh-extra-2, which adds a simple compilation fix.

These changes will (presumably) get pushed upstream for RethinkDB 2.5.

RethinkDB v2.4.0++ for Windows

This is an unofficial release of RethinkDB 2.4.0++, for Windows. The software is released under the Apache 2.0 license.

Suggestion

You might want to try running RethinkDB 2.4.2 (or later versions) on WSL 2. That is reported to work. Back when this Windows release was made, RethinkDB did not run successfully on WSL.

Limitations

This release does not support the js ReQL command.

The binary incorrectly reports its version number as v2.3.2-windows-beta-545-g85e842.

Installation

To run, download rethinkdb-v2.4.0-srh-win-1-Release_x64.zip, unzip it, and run the rethinkdb.exe binary. You don't need any of the other files.

Help & Support

You can read the official documentation for running on Windows.

Commercial support is available if you need help getting this running; email me or message me for more information.

You could also look for help on the Slack or Discord channels.

No Warranty

This release is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the release and assume any risks associated with it.

v2.4.0-srh-special-3

Another unofficial RethinkDB 2.4.0 RC release.

This includes unimportant changes not (yet?) in RethinkDB's official v2.4.x branch.

All Linux packages were built with ./setup.rb --v24support --commit fad2ca4bf --packages --copy-pkgs using the package builder as of commit https://github.com/srh/rethinkdb-package-builder/tree/e31404d53f035c24d86f11f7077ed1764ca1cbef .

The macOS package was built with make -j7 build-osx after applying the changes from 74510e2 to create_dmg.py. (I don't want to retag this RC yet again, sorry.)

JS Driver Test

Just a place to upload a test JavaScript driver. It holds the changes of the latest commit, which means everybody except the admin UI tries to use TcpConnection.

v2.4.0-srh-special-2

Another unofficial release candidate for 2.4. Includes some largely non-consequential changes.

If you want a package to be released for your platform, email me at sam@samuelhughes.com .

v2.3.6.srh.2

This is the same as v2.3.6.srh.1, and if you're already running that, there is no reason to upgrade. It has build fixes and packaging script fixes for newer platforms.

v2.4.0-srh-special-1

Made in order to have a name/place for the Bionic build of v2.4.x.

Consider this equivalent of not exactly a 2.4.0 release candidate, but something which is just... 2.4.x put into a Bionic package.

v2.3.6.srh.1

Carries some basic changes to get things building on new platforms, maybe other changes.

Instructions for Debian and Ubuntu users:
You probably don't want to install the -dbg versions of the .deb.

Run sudo dpkg -i foo.deb to install a deb package.

If you get a message about missing some dependencies, run sudo apt-get install -f to install them.

Then run rethinkdb --version to test that installation succeeded.

Instructions for users of other platforms:
Let me know if you want me to build and upload a package for your platform.

v2.3.6-srh-extra

This is a back-ported release of a soft durability flush interval feature.

Funding for this was provided by ZeroTier, a company that makes network virtualization software that allows networks to be created and managed effortlessly over both LAN and WAN.

This release is called v2.3.6-srh-extra. Its cluster protocol is not compatible with the official 2.3.6 binaries. The file format is, however, downgrade compatible. You can switch to this new version, try it out, and then switch back if you don't like it.

You may also upgrade from v2.3.5-srh-extra and keep your settings.

What's new? The same as in the previous v2.3.5-srh-extra release:

• Everything in the official 2.3.6 release is here
• Soft durability writes now get their data flushed every 5 seconds, instead of being flushed immediately. If you write and rewrite the same documents over and over again, you might see your I/O bandwidth decrease by a factor of 1000. Or more! (Or less.)
• A new field in the table configuration called 'user_value' has been added. It's initially set to the empty object, { }. You can do the following things with it:
  1. Set it to { "srh/flush_interval": 23.3 }. This will change the flush interval to 23.3 seconds, instead of 5 seconds.
  2. Set it to { "srh/flush_interval": 0.01 }, if you want your flushes to happen relatively quickly.
  3. Set it to { "srh/flush_interval": "default" }. This sets the flush interval to the default value, 5 seconds.
  4. Set it to { "srh/flush_interval": "never" }. This makes flushing "never" happen, under ideal conditions.
  5. Set it to whatever you like. Add new fields to the object for your own operational needs.

Under cache memory pressure, a flush could still happen sooner than the flush interval demands (even if set to "never"). Setting "never", or a very long flush interval, is only a good idea for tables that comfortably fit into memory. Otherwise, those tables will tend to hog memory that could be put to better use.

Example queries (in JavaScript) for configuring the flush interval of a table:

r.table('foo').config().update({'user_value': {'srh/flush_interval': 23.3}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 0.01}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 'default'}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 'never'}})

An example that sets the flush interval to 23.3 seconds, with some other user data:

r.table('foo').config().update({
    'user_value': {'srh/flush_interval': 23.3, 'john/blah': "john's data"}
})

It's possible to set the user value to whatever you want:
r.table('foo').config().update({'user_value': "hello world"})

To set the user value back to what it was initially:
r.table('foo').config().update({'user_value': { }})

You might also want to set the default durability to "soft":

r.table('foo').config().update({
    'durability': 'soft',
    'user_value': {'srh/flush_interval': 'never'}
})

Warning: If you set the user value, then downgrade to RethinkDB 2.3.6, and then upgrade back to 2.3.6-srh-extra, you might lose your user value configuration. (By design, you will lose your user value configuration if you modify the table's configuration at all while running RethinkDB 2.3.6.)

You might also notice some performance improvements in secondary index creation, bringing up new replicas, and running unit tests.

You might also notice performance regressions. Please report them to https://github.com/srh/rethinkdb/issues. My take is, if you're running the official RethinkDB 2.3.6, this release is far more likely to improve performance than hurt it, so it's worth using 2.3.6-srh-extra.

Binaries for Debian Jessie, Ubuntu 16.04, and CentOS 7 are available. Let me know if you'd like your platform on this list.

These changes will (presumably) get pushed upstream for RethinkDB 2.5.

v2.3.5-srh-extra

This is a back-ported release of a soft durability flush interval feature.

Funding for this was provided by ZeroTier, a company that makes network virtualization software that allows networks to be created and managed effortlessly over both LAN and WAN.

This release is called v2.3.5-srh-extra. Its cluster protocol is not compatible with the official 2.3.5 binaries. The file format is, however, downgrade compatible. You can switch to this new version, try it out, and then switch back if you don't like it.

What's new:

• Everything in the official 2.3.6 RC1 release is here
• Soft durability writes now get their data flushed every 5 seconds, instead of being flushed immediately. If you write and rewrite the same documents over and over again, you might see your I/O bandwidth decrease by a factor of 1000. Or more! (Or less.)
• A new field in the table configuration called 'user_value' has been added. It's initially set to the empty object, { }. You can do the following things with it:
  1. Set it to { "srh/flush_interval": 23.3 }. This will change the flush interval to 23.3 seconds, instead of 5 seconds.
  2. Set it to { "srh/flush_interval": 0.01 }, if you want your flushes to happen relatively quickly.
  3. Set it to { "srh/flush_interval": "default" }. This sets the flush interval to the default value, 5 seconds.
  4. Set it to { "srh/flush_interval": "never" }. This makes flushing "never" happen, under ideal conditions.
  5. Set it to whatever you like. Add new fields to the object for your own operational needs.

Under cache memory pressure, a flush could still happen sooner than the flush interval demands (even if set to "never"). Setting "never", or a very long flush interval, is only a good idea for tables that comfortably fit into memory. Otherwise, those tables will tend to hog memory that could be put to better use.

Example queries (in JavaScript) for configuring the flush interval of a table:

r.table('foo').config().update({'user_value': {'srh/flush_interval': 23.3}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 0.01}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 'default'}})
r.table('foo').config().update({'user_value': {'srh/flush_interval': 'never'}})

An example that sets the flush interval to 23.3 seconds, with some other user data:

r.table('foo').config().update({
    'user_value': {'srh/flush_interval': 23.3, 'john/blah': "john's data"}
})

It's possible to set the user value to whatever you want:
r.table('foo').config().update({'user_value': "hello world"})

To set the user value back to what it was initially:
r.table('foo').config().update({'user_value': { }})

You might also want to set the default durability to "soft":

r.table('foo').config().update({
    'durability': 'soft',
    'user_value': {'srh/flush_interval': 'never'}
})

Warning: If you set the user value, then downgrade to RethinkDB 2.3.5, and then upgrade back to 2.3.5-srh-extra, you might lose your user value configuration. (By design, you will lose your user value configuration if you modify the table's configuration at all while running RethinkDB 2.3.5.)

You might also notice some performance improvements in secondary index creation, bringing up new replicas, and running unit tests.

You might also notice performance regressions. Please report them to https://github.com/srh/rethinkdb/issues. My take is, if you're running the official RethinkDB 2.3.5, this release is far more likely to improve performance than hurt it, so it's worth using 2.3.5-srh-extra.

Binaries for Debian Jessie, Ubuntu 16.04, and CentOS 7 are coming soon. Let me know if you'd like your platform on this list.

These changes will (presumably) get pushed upstream for RethinkDB 2.5.