A small single file python utility for essentially splitting a redis database into a plethora of files with text (as opposed to binary) content. More specifically, the keys and corresponding content are (mostly) saved in base64_urlsafe encoding, which should eliminate most compatibility issues that could otherwise arise from illegal characters in file paths, file encoding differences, and it also means that the files could in theory be directly accessed via http(s). Furthermore, this means that both strings as well as binary data is supported without problems, although this means the actual data is encoded quite inefficiently. Essentially, the data generated by this script functions as a backup of all (or selected parts) of the content of your database, and subsequently the script can also restore the contents of your database from the files it has previously generated.
The dump is binary data, which makes it problematic for incremental backup solutions - or non-backup solutions that are misappropriated as such *cough*git*cough*.
This is a completely unofficial and badly tested script. It worked for me, but it might not work for you. Before using it, test that it works for you, and for the love of god copy your database before testing if the restore works. Even if it seems to work, don't solely rely on it, make regular backups of your database.
This script requires python3, and obviously access to a redis instance. Furthermore, we need the python redis package, which can be easily installed using pip:
pip install -r requirements.txt
python3 redis_load_store.py
The are a plethora of parameters available:
usage: redis_load_store.py [-h] [--host HOST] [--port PORT] [--unix_socket_path UNIX_SOCKET_PATH] [--password PASSWORD] [--db DB] [--empty]
[--use_expireat] [--keys KEYS]
{load,store,help} folder
Redis Text Backup Utility
positional arguments:
{load,store,help} Command to execute
folder Folder location for storing/loading data
options:
-h, --help show this help message and exit
Common Arguments:
--host HOST Host name or IP address for Redis server
--port PORT Port number for Redis server
--unix_socket_path UNIX_SOCKET_PATH
Path to the Unix socket for Redis connection
--password PASSWORD Password for Redis connection
--db DB Redis database to connect to
--empty Empty the datbase/folderbefore loading/storing the data
Load-specific Arguments:
--use_expireat (Missing) Use expireat instead of ttl when loading expiring keys
Store-specific Arguments:
--keys KEYS Only dump keys matching the specified pattern. Default: *
I haven't tested unix socket at all, and --use_expireat is not implemented. The expire timings are stored and loaded, but use the ttl, so relative timings to the point in time when the backup was made. So if a key only had 10s left to live when the backup was made, and you restore the backup a week later, it still has exactly 10s left before redis drops it.
./redis_load_store.py store backup_redis --empty
Stores the entire database 0 to the folder backup_redis, clears the folder before doing so (--empty).
./redis_load_store.py load backup_redis --empty
Loads the entire database 0from the folder backup_redis, clears the database beforehand (--empty).
If you deal with streams, not using --empty will result in (non-critical) errors being thrown.
I created an entry for each of the 6 types the script supports, then flushed all values in the redis instance, and subsequently used the load subcommand to restore the values.
The corresponding "test" folder looks as such:
- string
- list
- set
- zset
- hash
- stream
As all other types are just different views on the same data (such as bitmaps for strings) this should cover all types.
As I am myself not using streams I've only done a very rudimentary test so far, which has worked but no guarantee it does.
Tested with Python 3.11.2
Tested with Redis-Server 7.0.11
So far only tested on linux (debian 12), but other linux distros should work fine too, macos as well. In theory this should also work on windows, base64 url encode is used after all, but there's a potential issue with filename limits. While to some degree this affects all operating systems, windows has by default a comparatively small path size limit, which can be extended by changing settings. For more information: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry