You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The iterator in this code is a direct conversion of ArrayBuffer, not a real stream iterator, and in each call to getNextResultSet, there is no real call to Trino.client to get the next result each time. @hddong WDYT?
How should we improve?
Trino's proxy implementation should be based entirely on the incremental collect model.
Based on IterableFetchIterator to implement Trino DataResult Iterator, we can refer to Trino-JDBC's ResultSet implementation.
Are you willing to submit PR?
Yes I am willing to submit a PR!
The text was updated successfully, but these errors were encountered:
Code of Conduct
Search before asking
What would you like to be improved?
The advantage of the trino engine is the ability to stream returned data, kyuubi as a proxy should not remove this feature, storing the result set all in memory will lead to performance and OOM problems.
https://github.com/apache/incubator-kyuubi/blob/6a231519454c5d75aca2120de140c06febf7ae54/externals/kyuubi-trino-engine/src/main/scala/org/apache/kyuubi/engine/trino/TrinoStatement.scala#L81-L135
The iterator in this code is a direct conversion of ArrayBuffer, not a real stream iterator, and in each call to getNextResultSet, there is no real call to
Trino.client
to get the next result each time. @hddong WDYT?How should we improve?
IterableFetchIterator
to implement Trino DataResult Iterator, we can refer to Trino-JDBC'sResultSet
implementation.Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: