Crash updating state

[ds-testing-user]

We are a group of researchers testing distributed protocol implementations. While testing redisraft, we encountered the following crashes (with a 3 node cluster).

Crash 1

=== REDIS BUG REPORT START: Cut & paste starting from here ===
2058877:M 07 Jan 2024 20:05:41.782 # Redis 255.255.255 crashed by signal: 11, si_code: 1
2058877:M 07 Jan 2024 20:05:41.782 # Accessing address: (nil)
2058877:M 07 Jan 2024 20:05:41.782 # Crashed running the instruction at: 0x55a05025f9fc

------ STACK TRACE ------
EIP:
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5073(dictFind+0x6c)[0x55a05025f9fc]

Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x3bfd0)[0x7f8be6c75fd0]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5073(dictFind+0x6c)[0x55a05025f9fc]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5073(dictFetchValue+0x9)[0x55a05025fb09]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5073(RM_Call+0x172)[0x55a05034b6f2]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(+0x2a322)[0x7f8be5ba2322]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_become_follower+0x6f)[0x7f8be5bb33ff]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_accept_leader+0x80)[0x7f8be5bb68c0]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_recv_appendentries+0x10a)[0x7f8be5bb6d7a]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(handleTestNetworkMessage+0x11e)[0x7f8be5bad48e]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(poll_redis_test_messages+0x89)[0x7f8be5bad5c9]
/lib/x86_64-linux-gnu/libc.so.6(+0x89044)[0x7f8be6cc3044]
/lib/x86_64-linux-gnu/libc.so.6(+0x10961c)[0x7f8be6d4361c]

------ REGISTERS ------
2058877:M 07 Jan 2024 20:05:41.783 #
RAX:0000000000000000 RBX:00007f8be68090e0
RCX:00000000000000ff RDX:0000000000000000
RDI:1e6a9327948b3f7b RSI:3f8a3e48fdccdbe4
RBP:00007f8be686f700 RSP:00007f8bd99fd0f0
R8 :000000000000000a R9 :902511796a6b0e37
R10:4023b23bbd7d6b8e R11:00007f8bd8420043
R12:00007f8be6829780 R13:0000000000000100
R14:0000000000000000 R15:00007f8bd8420043
RIP:000055a05025f9fc EFL:0000000000010246
CSGSFS:002b000000000033
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0ff) -> 00000100d8413000
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0fe) -> 00007f8bd99fd3c0
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0fd) -> 00007f8bd99fd3c0
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0fc) -> 0000000000000100
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0fb) -> 000055a05034b6f2
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0fa) -> 00007f8be6829780
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f9) -> 000055a05025fb09
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f8) -> 00007f8bd841b000
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f7) -> 0000000000000000
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f6) -> 0000000000000100
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f5) -> 00007f8be6829780
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f4) -> 00007f8be686f700
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f3) -> 000055a050543aa0
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f2) -> 0000000000000004
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f1) -> de0d3d899f32f711
2058877:M 07 Jan 2024 20:05:41.783 # (00007f8bd99fd0f0) -> 000000009557e192

------ INFO OUTPUT ------

Server

redis_version:255.255.255
redis_git_sha1:e04ebdb8
redis_git_dirty:1
redis_build_id:1f78dcc7b4fbdd25
redis_mode:standalone
os:Linux 6.1.60.1.amd64-smp x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:2058877
process_supervised:no
run_id:72f8c23483ec34c9cf1b0b512eb251258114e70c
tcp_port:5073
server_time_usec:1704654341782997
uptime_in_seconds:2
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:10155525
executable:/home/ds-testing-user/Fuzzing/redis/src/redis-server
config_file:
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,port=5073

Clients

connected_clients:2
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:20534
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0

Memory

used_memory:1951864
used_memory_human:1.86M
used_memory_rss:18460672
used_memory_rss_human:17.61M
used_memory_peak:1965256
used_memory_peak_human:1.87M
used_memory_peak_perc:99.32%
used_memory_overhead:1771104
used_memory_startup:1726048
used_memory_dataset:180760
used_memory_dataset_perc:80.05%
allocator_allocated:2754592
allocator_active:3510272
allocator_resident:11751424
total_system_memory:270388695040
total_system_memory_human:251.82G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.27
allocator_frag_bytes:755680
allocator_rss_ratio:3.35
allocator_rss_bytes:8241152
rss_overhead_ratio:1.57
rss_overhead_bytes:6709248
mem_fragmentation_ratio:9.47
mem_fragmentation_bytes:16510648
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:44800
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

Persistence

loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:3
rdb_bgsave_in_progress:0
rdb_last_save_time:1704654339
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

Stats

total_connections_received:3
total_commands_processed:21
instantaneous_ops_per_sec:5
total_net_input_bytes:264
total_net_output_bytes:298
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.02
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:7
total_writes_processed:6
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:3
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0

Replication

role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:0762f8dabb1688c18da2b0d5387761dfaa03fa4e
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

CPU

used_cpu_sys:0.109354
used_cpu_user:0.048602
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.106437
used_cpu_user_main_thread:0.021287

Modules

module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]

Commandstats

cmdstat_hello:calls=1,usec=5,usec_per_call=5.00,rejected_calls=0,failed_calls=0
cmdstat_raft.cluster:calls=1,usec=4,usec_per_call=4.00,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=2,usec=3,usec_per_call=1.50,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1494,usec_per_call=747.00,rejected_calls=0,failed_calls=0
cmdstat_incr:calls=3,usec=12,usec_per_call=4.00,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=4,usec_per_call=1.33,rejected_calls=0,failed_calls=0
cmdstat_config|set:calls=1,usec=6,usec_per_call=6.00,rejected_calls=0,failed_calls=0

Errorstats

Latencystats

latency_percentiles_usec_hello:p50=5.023,p99=5.023,p99.9=5.023
latency_percentiles_usec_raft.cluster:p50=4.015,p99=4.015,p99.9=4.015
latency_percentiles_usec_auth:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_command:p50=745.471,p99=753.663,p99.9=753.663
latency_percentiles_usec_incr:p50=4.015,p99=4.015,p99.9=4.015
latency_percentiles_usec_config|get:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|set:p50=6.015,p99=6.015,p99.9=6.015

Cluster

cluster_enabled:0

Keyspace

db0:keys=1,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=8 addr=127.0.0.1:32946 laddr=127.0.0.1:5073 fd=9 name= age=2 idle=2 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=5 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=raft.cluster user=default redir=-1 resp=3 lib-name=go-redis(,go1.21.4) lib-ver=9.2.1
id=10 addr=127.0.0.1:32970 laddr=127.0.0.1:5073 fd=10 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22400 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=

------ MODULES INFO OUTPUT ------

raft_version

raft_version:255.255.255
raft_git_sha1:b779b199

raft_general

raft_dbid:7542e419a20589a9b22739c12813f921
raft_node_id:3
raft_state:up
raft_role:follower
raft_is_voting:no
raft_voted_for:-1
raft_leader_id:-1
raft_current_term:2
raft_num_nodes:3
raft_num_voting_nodes:1
raft_node1:id=1,state=connected,voting=yes,addr=localhost,port=5071,last_conn_secs=0,conn_errors=11,conn_oks=2
raft_node2:id=2,state=connect_error,voting=no,addr=localhost,port=5072,last_conn_secs=2,conn_errors=13,conn_oks=1

raft_log

raft_log_entries:8
raft_current_index:8
raft_commit_index:8
raft_last_applied_index:8
raft_file_size:1587
raft_cache_memory_size:1345
raft_cache_entries:8
raft_client_attached_entries:0
raft_fsync_count:5
raft_fsync_max_microseconds:8
raft_entry1:term=1,id=0,data_len=0
raft_entry2:term=1,id=136837437,data_len=264
raft_entry3:term=1,id=1836619709,data_len=264
raft_entry4:term=1,id=1523149435,data_len=264
raft_entry5:term=1,id=145366382,data_len=35
raft_entry6:term=1,id=726943297,data_len=35
raft_entry7:term=2,id=0,data_len=0
raft_fsync_avg_microseconds:6

raft_snapshot

raft_snapshot_filename:redis3.rdb
raft_snapshot_last_idx:0
raft_snapshot_last_term:0
raft_snapshot_size:0
raft_snapshot_time_secs:-1
raft_snapshots_created:0
raft_snapshots_received:0
raft_snapshot_in_progress:no
raft_snapshot_in_progress_last_idx:-1
raft_snapshot_in_progress_last_term:-1

raft_clients

raft_proxy_reqs:0
raft_proxy_failed_reqs:0
raft_proxy_failed_responses:0
raft_proxy_outstanding_reqs:0

raft_stats

raft_appendreq_received:0
raft_appendreq_with_entry_received:0
raft_snapshotreq_received:0
raft_exec_throttled:0
raft_num_sessions:0

------ CONFIG DEBUG OUTPUT ------
replica-read-only yes
slave-read-only yes
lazyfree-lazy-user-flush no
io-threads 1
repl-diskless-sync yes
lazyfree-lazy-server-del no
io-threads-do-reads no
sanitize-dump-payload no
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
client-query-buffer-limit 1gb
lazyfree-lazy-user-del no
repl-diskless-load disabled
proto-max-bulk-len 512mb
list-compress-depth 0
activedefrag no

------ FAST MEMORY TEST ------
2058877:M 07 Jan 2024 20:05:41.783 # main thread terminated
2058877:M 07 Jan 2024 20:05:41.784 # Bio worker thread #0 terminated
2058877:M 07 Jan 2024 20:05:41.784 # Bio worker thread #1 terminated
2058877:M 07 Jan 2024 20:05:41.784 # Bio worker thread #2 terminated

Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: dictFind (base: 0x55a05025f990)
Module: /home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5073 (base 0x55a0501d9000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x55a05025f990 -D -b binary -m i386:x86-64 /tmp/dump.bin

2058877:M 07 Jan 2024 20:05:41.784 # dump of function (hexdump of 236 bytes):
415741564155415455534883ec18488b4720480347180f84c500000048837f28ff4889fb4989f7740b66837f30000f841c010000488b034c89ff4531f6ff104889442408420fb64c333231c080f9ff741648c7c0ffffffff48d3e048f7d0482344240848c1e0034a8b54f3084c8b24024d85e4745b4589e54489e54183e501757f4c89e083e0074883f8020f848f000000498b14244939d7744a488b03488b40184885c0740c4c89fe4889dfffd085c075324585ed75194c89e083e0074883f80274454d8b6424104d85e475a80f1f0048837b28ff740a4983fe010f857e0000004531e44883c4184c89e05b

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

Crash 2

=== REDIS BUG REPORT START: Cut & paste starting from here ===
3768014:M 01 Jan 2024 21:34:47.118 # Redis 255.255.255 crashed by signal: 6, si_code: -6
3768014:M 01 Jan 2024 21:34:47.118 # Crashed running the instruction at: 0x7f463d603d3c

------ STACK TRACE ------
EIP:
/lib/x86_64-linux-gnu/libc.so.6(+0x8ad3c)[0x7f463d603d3c]

Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x3bfd0)[0x7f463d5b4fd0]
/lib/x86_64-linux-gnu/libc.so.6(+0x8ad3c)[0x7f463d603d3c]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x12)[0x7f463d5b4f32]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7f463d59f472]
/lib/x86_64-linux-gnu/libc.so.6(+0x26395)[0x7f463d59f395]
/lib/x86_64-linux-gnu/libc.so.6(+0x34e32)[0x7f463d5ade32]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_handle_append_cfg_change+0x2c2)[0x7f463d528942]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_restore_log+0xb2)[0x7f463d52c932]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(callRaftPeriodic+0x131)[0x7f463d518d41]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(moduleTimerHandler+0x21c)[0x55e0d297d98c]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x81705)[0x55e0d2887705]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(aeMain+0x1c8)[0x55e0d2891cb8]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(main+0x3cd)[0x55e0d2886e2d]
/lib/x86_64-linux-gnu/libc.so.6(+0x271ca)[0x7f463d5a01ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f463d5a0285]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(_start+0x2a)[0x55e0d28875aa]

------ REGISTERS ------
3768014:M 01 Jan 2024 21:34:47.119 #
RAX:0000000000000000 RBX:0000000000397ece
RCX:00007f463d603d3c RDX:0000000000000006
RDI:0000000000397ece RSI:0000000000397ece
RBP:00007f463d5751c0 RSP:00007ffe7ca29a50
R8 :0000000000000007 R9 :000055e0d4ccf610
R10:0000000000000008 R11:0000000000000246
R12:0000000000000006 R13:0000000000000117
R14:00007f463d54c1df R15:00007ffe7ca29d00
RIP:00007f463d603d3c EFL:0000000000000246
CSGSFS:002b000000000033
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a5f) -> 000055e0d4cc9e80
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a5e) -> 0000000000000007
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a5d) -> 0000000000000012
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a5c) -> 000055e0d4ccf610
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a5b) -> 0000000000000117
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a5a) -> 0000000000000020
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a59) -> 00007f463d59f472
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a58) -> 00007f463d74ce70
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a57) -> 00007f463d5b4f32
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a56) -> 00007f463d54c1df
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a55) -> 0000000000000117
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a54) -> 00007f463d554720
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a53) -> 00007f463d5751c0
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a52) -> 0000000000000006
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a51) -> 615bcbb021614a00
3768014:M 01 Jan 2024 21:34:47.119 # (00007ffe7ca29a50) -> 00007ffe7ca29d00

------ INFO OUTPUT ------

Server

redis_version:255.255.255
redis_git_sha1:e04ebdb8
redis_git_dirty:1
redis_build_id:1f78dcc7b4fbdd25
redis_mode:standalone
os:Linux 6.1.60.1.amd64-smp x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:3768014
process_supervised:no
run_id:a8f4b63f4d387de775174b6672350f64e7be881e
tcp_port:5043
server_time_usec:1704141287117542
uptime_in_seconds:1
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:9642471
executable:/home/ds-testing-user/Fuzzing/redis/src/redis-server
config_file:
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,port=5043

Clients

connected_clients:0
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0

Memory

used_memory:1755224
used_memory_human:1.67M
used_memory_rss:17649664
used_memory_rss_human:16.83M
used_memory_peak:1755224
used_memory_peak_human:1.67M
used_memory_peak_perc:100.49%
used_memory_overhead:1746728
used_memory_startup:1746544
used_memory_dataset:8496
used_memory_dataset_perc:97.88%
allocator_allocated:2165200
allocator_active:2703360
allocator_resident:10866688
total_system_memory:270388695040
total_system_memory_human:251.82G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.25
allocator_frag_bytes:538160
allocator_rss_ratio:4.02
allocator_rss_bytes:8163328
rss_overhead_ratio:1.62
rss_overhead_bytes:6782976
mem_fragmentation_ratio:10.10
mem_fragmentation_bytes:15902976
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

Persistence

loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1704141286
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

Stats

total_connections_received:0
total_commands_processed:6
instantaneous_ops_per_sec:23
total_net_input_bytes:0
total_net_output_bytes:0
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:0
total_writes_processed:0
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0

Replication

role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:9f37188d7e2696298b5cd2ef6e4e7378485a4f66
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

CPU

used_cpu_sys:0.009746
used_cpu_user:0.009746
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.009233
used_cpu_user_main_thread:0.009233

Modules

module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]

Commandstats

cmdstat_config|set:calls=1,usec=11,usec_per_call=11.00,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=4,usec_per_call=1.33,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1642,usec_per_call=821.00,rejected_calls=0,failed_calls=0

Errorstats

Latencystats

latency_percentiles_usec_config|set:p50=11.007,p99=11.007,p99.9=11.007
latency_percentiles_usec_config|get:p50=2.007,p99=2.007,p99.9=2.007
latency_percentiles_usec_command:p50=761.855,p99=884.735,p99.9=884.735

Cluster

cluster_enabled:0

Keyspace

------ CLIENT LIST OUTPUT ------

------ MODULES INFO OUTPUT ------

raft_version

raft_version:255.255.255
raft_git_sha1:b779b199

raft_general

raft_dbid:318c057497ad2a53d41fc4fa17aedeb9
raft_node_id:3
raft_state:up
raft_role:follower
raft_is_voting:yes
raft_voted_for:-1
raft_leader_id:-1
raft_current_term:0
raft_num_nodes:2
raft_num_voting_nodes:1
raft_node1:id=2,state=disconnected,voting=no,addr=localhost,port=5042,last_conn_secs=-1,conn_errors=0,conn_oks=0

raft_log

raft_log_entries:6
raft_current_index:6
raft_commit_index:0
raft_last_applied_index:0
raft_file_size:1637
raft_cache_memory_size:0
raft_cache_entries:0
raft_client_attached_entries:0
raft_fsync_count:0
raft_fsync_max_microseconds:0
raft_entry1:term=1,id=0,data_len=0
raft_entry2:term=1,id=298115813,data_len=264
raft_entry3:term=1,id=269428799,data_len=264
raft_entry4:term=1,id=680133272,data_len=264
raft_entry5:term=1,id=601065526,data_len=35
raft_fsync_avg_microseconds:0

raft_snapshot

raft_snapshot_filename:redis3.rdb
raft_snapshot_last_idx:0
raft_snapshot_last_term:0
raft_snapshot_size:0
raft_snapshot_time_secs:-1
raft_snapshots_created:0
raft_snapshots_received:0
raft_snapshot_in_progress:no
raft_snapshot_in_progress_last_idx:-1
raft_snapshot_in_progress_last_term:-1

raft_clients

raft_proxy_reqs:0
raft_proxy_failed_reqs:0
raft_proxy_failed_responses:0
raft_proxy_outstanding_reqs:0

raft_stats

raft_appendreq_received:0
raft_appendreq_with_entry_received:0
raft_snapshotreq_received:0
raft_exec_throttled:0
raft_num_sessions:0

------ CONFIG DEBUG OUTPUT ------
slave-read-only yes
lazyfree-lazy-eviction no
io-threads-do-reads no
activedefrag no
io-threads 1
lazyfree-lazy-user-flush no
list-compress-depth 0
sanitize-dump-payload no
repl-diskless-sync yes
lazyfree-lazy-expire no
repl-diskless-load disabled
lazyfree-lazy-server-del no
lazyfree-lazy-user-del no
proto-max-bulk-len 512mb
client-query-buffer-limit 1gb
replica-read-only yes

------ FAST MEMORY TEST ------
3768014:M 01 Jan 2024 21:34:47.119 # Bio worker thread #0 terminated
3768014:M 01 Jan 2024 21:34:47.120 # Bio worker thread #1 terminated
3768014:M 01 Jan 2024 21:34:47.120 # Bio worker thread #2 terminated
*** Preparing to test memory region 55e0d2b5d000 (2273280 bytes)
*** Preparing to test memory region 55e0d4c7a000 (405504 bytes)
*** Preparing to test memory region 7f4624000000 (135168 bytes)
*** Preparing to test memory region 7f462b800000 (8388608 bytes)
*** Preparing to test memory region 7f462c000000 (135168 bytes)
*** Preparing to test memory region 7f46307ff000 (8388608 bytes)
*** Preparing to test memory region 7f4631000000 (8388608 bytes)
*** Preparing to test memory region 7f4631800000 (6291456 bytes)
*** Preparing to test memory region 7f4631ffd000 (8388608 bytes)
*** Preparing to test memory region 7f46327fe000 (8388608 bytes)
*** Preparing to test memory region 7f4632fff000 (8388608 bytes)
*** Preparing to test memory region 7f4633800000 (8388608 bytes)
*** Preparing to test memory region 7f4634000000 (135168 bytes)
*** Preparing to test memory region 7f46382dc000 (8388608 bytes)
*** Preparing to test memory region 7f4638add000 (8388608 bytes)
*** Preparing to test memory region 7f46392de000 (8388608 bytes)
*** Preparing to test memory region 7f4639adf000 (8388608 bytes)
*** Preparing to test memory region 7f463a2e0000 (8388608 bytes)
*** Preparing to test memory region 7f463aae1000 (8388608 bytes)
*** Preparing to test memory region 7f463b2e1000 (2621440 bytes)
*** Preparing to test memory region 7f463b570000 (8192 bytes)
*** Preparing to test memory region 7f463bc7f000 (12288 bytes)
*** Preparing to test memory region 7f463c218000 (16384 bytes)
*** Preparing to test memory region 7f463c29e000 (8192 bytes)
*** Preparing to test memory region 7f463cc00000 (8388608 bytes)
*** Preparing to test memory region 7f463d419000 (4096 bytes)
*** Preparing to test memory region 7f463d4e9000 (12288 bytes)
*** Preparing to test memory region 7f463d56e000 (45056 bytes)
*** Preparing to test memory region 7f463d74d000 (53248 bytes)
*** Preparing to test memory region 7f463d86a000 (8192 bytes)
.3768014:M 01 Jan 2024 21:34:47.124 * State change: Node is now a follower, term 1
3768014:M 01 Jan 2024 21:34:47.124 # Redis 255.255.255 crashed by signal: 11, si_code: 1
3768014:M 01 Jan 2024 21:34:47.124 # Accessing address: 0x55e0d2e6c220
3768014:M 01 Jan 2024 21:34:47.124 # Crashed running the instruction at: 0x55e0d2a54bd5

------ STACK TRACE ------
EIP:
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x24ebd5)[0x55e0d2a54bd5]

Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x3bfd0)[0x7f463d5b4fd0]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x24ebd5)[0x55e0d2a54bd5]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x25539b)[0x55e0d2a5b39b]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x255699)[0x55e0d2a5b699]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x25feee)[0x55e0d2a65eee]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x260080)[0x55e0d2a66080]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x25f0cd)[0x55e0d2a650cd]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x20894f)[0x55e0d2a0e94f]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x25d09b)[0x55e0d2a6309b]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x26e6e3)[0x55e0d2a746e3]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x26ed98)[0x55e0d2a74d98]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(+0x270973)[0x55e0d2a76973]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(je_calloc+0x1da)[0x55e0d2a04c1a]
/home/ds-testing-user/Fuzzing/redis/src/redis-server 0.0.0.0:5043(RM_Calloc+0x33)[0x55e0d2969473]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(+0x2a178)[0x7f463d516178]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(+0x2a2d0)[0x7f463d5162d0]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_become_follower+0x6f)[0x7f463d5273ff]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(raft_recv_requestvote_response+0x230)[0x7f463d52a800]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(handleTestNetworkMessage+0x15f)[0x7f463d5214cf]
/home/ds-testing-user/Fuzzing/redisraft-fuzzing/redisraft.so(poll_redis_test_messages+0x89)[0x7f463d5215c9]
/lib/x86_64-linux-gnu/libc.so.6(+0x89044)[0x7f463d602044]
/lib/x86_64-linux-gnu/libc.so.6(+0x10961c)[0x7f463d68261c]

------ REGISTERS ------
3768014:M 01 Jan 2024 21:34:47.125 #
RAX:000055e0d2e6c328 RBX:0000000000000000
RCX:000055e0d2e6c328 RDX:0000000000000070
RDI:00007f4600000000 RSI:00007f462bffeee0
RBP:00007f463d1c1000 RSP:00007f462bffd920
R8 :00007f4600000000 R9 :0000000000000001
R10:000055e0d2c83418 R11:0000000000022000
R12:00007f462bffec20 R13:000055e0d2b84ae0
R14:000055e0d2e6c220 R15:00007f462bffede0
RIP:000055e0d2a54bd5 EFL:0000000000010206
CSGSFS:002b000000000033
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd92f) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd92e) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd92d) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd92c) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd92b) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd92a) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd929) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd928) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd927) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd926) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd925) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd924) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd923) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd922) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd921) -> 0000000000000000
3768014:M 01 Jan 2024 21:34:47.125 # (00007f462bffd920) -> 0000000000000000

------ INFO OUTPUT ------
3768014:signal-handler (1704141287) Received SIGINT scheduling shutdown...

----- Stderr -----
libgcov profiling error:/home/ds-testing-user/Fuzzing/redisraft-fuzzing/build/deps/raft/CMakeFiles/raft.dir/src/raft_server.c.gcda:Merge mismatch for function 50
redis-server: /home/ds-testing-user/Fuzzing/redisraft-fuzzing/deps/raft/src/raft_server.c:279: raft_handle_append_cfg_change: Assertion `node' failed.