Skip to content

Avoid nested JSON elements in ps_crud #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 11, 2025
Merged

Avoid nested JSON elements in ps_crud #108

merged 1 commit into from
Jul 11, 2025

Conversation

simolus3
Copy link
Contributor

To collect local writes, we use json_object() to combine column values into a CRUD entry. A problem with this is that, if one of the columns has a JSON subtype, json_object() writes it as a nested object instead of a string. Following this suggestion, we wrap text column types in a single-argument concat call to drop the subtype.

I have not yet checked the performance impact of this. Another approach that unfortunately occurred too late to me may be to re-implement json_object in PowerSync, avoiding the nested behavior we don't need. That could potentially be faster.

@simolus3 simolus3 requested a review from rkistner July 11, 2025 08:30
Copy link
Contributor

@rkistner rkistner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some very basic performance tests, comparing the performance of these two queries:

select sum(length(json_object('t', data))), count() from ps_oplog;
select sum(length(json_object('t', concat(data)))), count() from ps_oplog;

In these cases, data is always a string, so I'm just measuring the overhead when no actual conversion is required.

In my test set, I have 114k rows, totaling around 100MiB in size. The first version takes an average of 117ms, while the concat version takes an average of 120ms. And this is a large data set, much larger than typical data we'd see in the upload queue. Compared to the rest of the overhead we have in the upload queue, including actually writing the data, this overhead appears to be negligible.

@simolus3 simolus3 merged commit b257451 into main Jul 11, 2025
21 checks passed
@simolus3 simolus3 deleted the avoid-nested-json branch July 11, 2025 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants