When connection times out a SIGSEGV is encountered

[warmwaffles]

For a large table of 66,000 records

Company
|> select([c], c)
|> Repo.all()

Encounter

14:25:34.215 [error] Exqlite.Connection (#PID<0.446.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.536.0> timed out because it queued and checked out the connection for longer than 15000ms

#PID<0.536.0> was at location:

    (exqlite 0.5.1) lib/exqlite/sqlite3.ex:107: Exqlite.Sqlite3.fetch_all/3
    (exqlite 0.5.1) lib/exqlite/connection.ex:486: Exqlite.Connection.get_rows/2
    (exqlite 0.5.1) lib/exqlite/connection.ex:439: Exqlite.Connection.execute/4
    (db_connection 2.3.1) lib/db_connection/holder.ex:316: DBConnection.Holder.holder_apply/4
    (db_connection 2.3.1) lib/db_connection.ex:1272: DBConnection.run_execute/5
    (db_connection 2.3.1) lib/db_connection.ex:1359: DBConnection.run/6
    (db_connection 2.3.1) lib/db_connection.ex:557: DBConnection.parsed_prepare_execute/5
    (db_connection 2.3.1) lib/db_connection.ex:550: DBConnection.prepare_execute/4
    (ecto_sqlite3 0.5.3) lib/ecto/adapters/sqlite3/connection.ex:62: Ecto.Adapters.SQLite3.Connection.prepare_execute/5
    (ecto_sql 3.5.4) lib/ecto/adapters/sql.ex:692: Ecto.Adapters.SQL.execute!/4
    (ecto_sql 3.5.4) lib/ecto/adapters/sql.ex:684: Ecto.Adapters.SQL.execute/5
    (ecto 3.5.8) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
    (ecto 3.5.8) lib/ecto/repo/queryable.ex:17: Ecto.Repo.Queryable.all/3
    (stdlib 3.14) erl_eval.erl:680: :erl_eval.do_apply/6
    (elixir 1.11.3) src/elixir.erl:280: :elixir.recur_eval/3
    (elixir 1.11.3) src/elixir.erl:265: :elixir.eval_forms/3
    (iex 1.11.3) lib/iex/evaluator.ex:261: IEx.Evaluator.handle_eval/5
    (iex 1.11.3) lib/iex/evaluator.ex:242: IEx.Evaluator.do_eval/3
    (iex 1.11.3) lib/iex/evaluator.ex:220: IEx.Evaluator.eval/3

fish: Job 1, 'iex -S mix' terminated by signal SIGSEGV (Address boundary error)

This may be a reason why esqlite added bulk fetching rows instead of simply stepping. Although bulk fetching would be nice, the issue arises that an error happens midway through the step, and how to communicate that error back is a bit tricky.

[warmwaffles]

We do multi-stepping when fetching. It was simply too slow to fetch individual rows and return back to vm.

[warmwaffles]

exqlite/lib/exqlite/sqlite3.ex

Lines 119 to 149 in 0008856

	@spec fetch_all(db(), statement(), integer()) :: {:ok, [row()]} | {:error, reason()}
	def fetch_all(conn, statement, chunk_size) do
	fetch_all(conn, statement, chunk_size, [])
	end
	@spec fetch_all(db(), statement()) :: {:ok, [row()]} | {:error, reason()}
	def fetch_all(conn, statement) do
	# Should this be done in the NIF? It can be _much_ faster to build a list
	# there, but at the expense that it could block other dirty nifs from
	# getting work done.
	#
	# For now this just works
	chunk_size = Application.get_env(:exqlite, :default_chunk_size, 50)
	fetch_all(conn, statement, chunk_size, [])
	end
	defp fetch_all(conn, statement, chunk_size, accum) do
	case multi_step(conn, statement, chunk_size) do
	{:done, rows} ->
	{:ok, accum ++ rows}
	{:rows, rows} ->
	fetch_all(conn, statement, chunk_size, accum ++ rows)
	{:error, reason} ->
	{:error, reason}
	:busy ->
	{:error, "Database busy"}
	end
	end

which is called from

exqlite/lib/exqlite/connection.ex

Lines 565 to 573 in 0008856

	defp get_rows(%Query{ref: ref, statement: statement}, state) do
	case Sqlite3.fetch_all(state.db, ref, state.chunk_size) do
	{:ok, rows} ->
	{:ok, rows}
	{:error, reason} ->
	{:error, %Error{message: reason, statement: statement}, state}
	end
	end

and consumed by

exqlite/lib/exqlite/connection.ex

Lines 509 to 514 in 0008856

	defp execute(call, %Query{} = query, params, state) do
	with {:ok, query} <- bind_params(query, params, state),
	{:ok, columns} <- get_columns(query, state),
	{:ok, rows} <- get_rows(query, state),
	{:ok, transaction_status} <- Sqlite3.transaction_status(state.db),
	changes <- maybe_changes(state.db, query) do

[LostKobrakai]

@warmwaffles I'm seeing a segfault while the NIF is running multi_step (#191) when the outer db connection timeouts. So something seems to still be off, even with returning things to elixir in small chunks.

[warmwaffles]

@LostKobrakai were you able to change the chunk size and still get the same error? Just trying to collect as much info as I can.

[warmwaffles]

Also, how big do you think on average each one of those 66,000 records are?

[LostKobrakai]

I tried running the test in the PR I linked with chunk_size 1, 10, 1000, 10000 and it succeeded for the latter two. But increasing the number of records from 10k to 50k made it segfault again for all chunk sizes. It might just get fast enough with a larger chunk size, but still fail when hitting the timeout.

[warmwaffles]

Okay, when I get some time today I'll dig into this more.

[LostKobrakai]

My PR should replicate the issue on the latest version. But yes I was confused why you reacted to the old thread as well :D

[warmwaffles]

Hah, yea completely forgot I made this ticket to begin with. I had a case where I had a ton of time series data stored and retrieving it was difficult.