Poor performance of Multi Bulk response parser

[epsylonix]

From the documentation I understand that there is a performance optimization for multibulk response processing that sets eredis apart from other libraries. It seems that (some?) multibulk responses are actually processed orders of magnitude slower than non-multibulk responses. Here is a sample code to reproduce poor performance for multibulk response parsing:

> {ok, E} = eredis:start_link().
> eredis:q(E,["FLUSHDB"]).

> A = binary:copy(<<"a">>, 10000000).
> eredis:q(E,["HSET", "h", "1", A]).

> timer:tc(fun() -> :eredis.q(e,["HGET", "h", "1"]) end).  
{58671, {ok, <<"aaa"...>>}}

> timer:tc(fun() -> :eredis.q(e,["HGETALL", "h"], 100000) end).  
> {16966651, {ok,  [<<"1">>, <<"aaa"...>>]}}

The size of the returned data is almost the same but the first command resulted in a RESP Bulk String response and the second one returned a RESP Array (multibulk) response. The first command was executed in 58ms, the second took almost 17 seconds!

[zuiderkwast]

Yes, it's the parser. Some profiling using cprof:

1> cprof:start(), R=eredis:q(E, ["HGETALL", "h"], 100000), cprof:pause(),R.
{ok,[<<"1">>,
     <<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...>>]}
2> cprof:analyse(eredis_parser).
{eredis_parser,82216,
               [{{eredis_parser,buffer_to_binary,1},6853},
                {{eredis_parser,buffer_create,1},6853},
                {{eredis_parser,get_newline_pos,1},6852},
                {{eredis_parser,do_parse_multibulk,3},6852},
                {{eredis_parser,parse_bulk,1},6851},
                {{eredis_parser,do_parse_bulk,1},6851},
                {{eredis_parser,buffer_size,1},6851},
                {{eredis_parser,buffer_hd,1},6851},
                {{eredis_parser,return_result,3},6850},
                {{eredis_parser,parse,2},6850},
                {{eredis_parser,parse_multibulk,2},6849},
                {{eredis_parser,buffer_append,2},6849},
                {{eredis_parser,parse_multibulk,1},2},
                {{eredis_parser,init,0},1},
                {{eredis_parser,do_parse_multibulk,2},1}]}
3> cprof:stop().
16107
4> f(R).
5> cprof:start(), R=eredis:q(E, ["HGET", "h", 1], 100000), cprof:pause(),R.
{ok,<<"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"...>>}
6> cprof:analyse(eredis_parser).                                           
{eredis_parser,34257,
               [{{eredis_parser,return_result,3},6850},
                {{eredis_parser,parse,2},6850},
                {{eredis_parser,parse_bulk,2},6849},
                {{eredis_parser,buffer_size,1},6849},
                {{eredis_parser,buffer_append,2},6849},
                {{eredis_parser,parse_bulk,1},2},
                {{eredis_parser,buffer_to_binary,1},2},
                {{eredis_parser,buffer_create,1},2},
                {{eredis_parser,init,0},1},
                {{eredis_parser,get_newline_pos,1},1},
                {{eredis_parser,do_parse_bulk,1},1},
                {{eredis_parser,buffer_hd,1},1}]}
7> cprof:stop().                                                           
16107