Change messages_data join table indexes for data_id lookup #653
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In long runs, we can see this query becomes very inefficient when a large amount of data has built up in the system:
firefly/internal/events/aggregator.go
Lines 609 to 618 in b7937f7
The original intention of this index was it would add a some (unnecessary in my view at this point) additional protection against data ID uniqueness within a message (already enforced strongly in code), while allowing a lookup in either direction to be efficient:
firefly/db/migrations/postgres/000003_create_message_data_join_table.up.sql
Line 9 in b7937f7
However, the results clearly indicate to me it's only helping with lookup in the
message_idbased lookups - and not thedata_idbased lookups. So this PR proposes we replace it with two separate and simple indexes.