`rewind to the marker` failure

[lejmr]

Hi,

I am sending medium (about 20 inserts per second) amount of rows into QuestDB instance, and I am getting following error. Any suggestion?

Exception in thread xxx-processing-thread-140173359330640:
Traceback (most recent call last):
  File "src/questdb/ingress.pyx", line 650, in questdb.ingress.Buffer._row
  File "src/questdb/ingress.pyx", line 618, in questdb.ingress.Buffer._at
  File "src/questdb/ingress.pyx", line 613, in questdb.ingress.Buffer._at_now
  File "src/questdb/ingress.pyx", line 592, in questdb.ingress.Buffer._may_trigger_row_complete
  File "src/questdb/ingress.pyx", line 329, in questdb.ingress.may_flush_on_row_complete
  File "src/questdb/ingress.pyx", line 1288, in questdb.ingress.Sender.flush
  File "src/questdb/ingress.pyx", line 1278, in questdb.ingress.Sender.flush
questdb.ingress.IngressError: Could not flush buffer: Broken pipe (os error 32)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/venv/lib/python3.9/site-packages/library/interface/handlers/abstract_solace.py", line 365, in _queue_lift_and_run
    self.process_inbound_message(inbound_message)
  File "/venv/lib/python3.9/site-packages/library/interface/exception_handling.py", line 53, in wrapper
    raise e
  File "/venv/lib/python3.9/site-packages/library/interface/exception_handling.py", line 44, in wrapper
    return func(*args, **kwargs)
  File "/venv/lib/python3.9/site-packages/library/interface/handlers/abstract_solace.py", line 141, in process_inbound_message
    self.process_data_update(incoming_data=incoming_data)
  File "/usr/local/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/venv/lib/python3.9/site-packages/library/interface/handlers/abstract.py", line 115, in process_data_update
    self.data_catalog_listener.on_update(self.data_catalog, update_identification)
  File "/venv/lib/python3.9/site-packages/library/services/calculation/single_thread.py", line 14, in on_update
    return self._execution_wrapper.execute(data_subset, content_id)
  File "/usr/local/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/venv/lib/python3.9/site-packages/library/interface/exception_handling.py", line 53, in wrapper
    raise e
  File "/venv/lib/python3.9/site-packages/library/interface/exception_handling.py", line 44, in wrapper
    return func(*args, **kwargs)
  File "/venv/lib/python3.9/site-packages/library/services/calculation/base.py", line 99, in execute
    process_response(response, data_catalog, self)
  File "/usr/local/lib/python3.9/functools.py", line 888, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/venv/lib/python3.9/site-packages/library/services/post_calculation/solace_archive.py", line 30, in _
    execution_wrapper.questdb_archiver.insert(table=obj.series, symbols=symbols, columns=columns)
  File "/venv/lib/python3.9/site-packages/library/interface/connectors/questdb.py", line 96, in insert
    self._sender.row(
  File "src/questdb/ingress.pyx", line 1237, in questdb.ingress.Sender.row
  File "src/questdb/ingress.pyx", line 719, in questdb.ingress.Buffer.row
  File "src/questdb/ingress.pyx", line 653, in questdb.ingress.Buffer._row
  File "src/questdb/ingress.pyx", line 490, in questdb.ingress.Buffer._rewind_to_marker
questdb.ingress.IngressError: Can't rewind to the marker: No marker set.

[amunra]

The "Broken Pipe" part is the real error: The TCP connection was dropped.

The second error (the marker cleanup) is an internal post-disconnect cleanup error that will be fixed in the next release (thanks for the find!), but is irrelevant to your actual issue.

Client/server disconnects may happen for a number of reasons, but often it is due to invalid data. The client library will validate a number of data issues in advance, but it will not catch all of them.
If the server detects invalid input it will disconnect to avoid further invalid input. I suspect this is what happened.

The best thing to do is to dig through the server logs to debug the issue.

See: https://py-questdb-client.readthedocs.io/en/latest/troubleshooting.html#inspecting-and-debugging-errors

[lejmr]

Thanks yes, the problem was on the table level where I had a mismatch of types.. i.e., table had string column where I was sending a timestamp data.