Skip to content

fix: improve performance on /account/balance for addresses with large transactions count (low effort improvements) #402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

matiwinnetou
Copy link
Collaborator

@matiwinnetou matiwinnetou commented Apr 9, 2025

This PR tries to do some "quick wins" without actually denormalising the data or changing the query, more drastic approaches we can try out post 1.2.7 release version.

POST /account/balance
{
    "network_identifier": {
        "blockchain": "cardano",
        "network": "mainnet"
    },
    "account_identifier": {
        "address": "addr1v804tgee0m3ww7z93zh64wr9flqh9psdhnxg6cykfudgulg6f633p",
        "metadata": {}
    }
}
2025-04-09 16:01:59.674 UTC [4055] LOG:  duration: 1599.280 ms  plan:
        Query Text: select aue1_0.output_index,aue1_0.tx_hash,aue1_0.amounts,aue1_0.block,aue1_0.owner_addr,aue1_0.owner_stake_addr from address_utxo aue1_0 where aue1_0.owner_addr=$1 and not exists(select 1 from tx_input tie1_0 where tie1_0.tx_hash=aue1_0.tx_hash and tie1_0.output_index=aue1_0.output_index and tie1_0.spent_at_block<=$2) and aue1_0.block<=$3
        Gather  (cost=5513.64..523753.79 rows=277 width=474) (actual time=1368.904..1599.253 rows=5 loops=1)
          Workers Planned: 6
          Workers Launched: 6
          Buffers: shared hit=2297508
          ->  Nested Loop Anti Join  (cost=5313.64..523539.94 rows=46 width=474) (actual time=1345.803..1345.806 rows=1 loops=7)
                Buffers: shared hit=2297508
                ->  Parallel Bitmap Heap Scan on address_utxo aue1_0  (cost=5312.95..472751.51 rows=67964 width=474) (actual time=301.092..524.717 rows=54550 loops=7)
                      Recheck Cond: ((owner_addr)::text = 'addr1v804tgee0m3ww7z93zh64wr9flqh9psdhnxg6cykfudgulg6f633p'::text)
                      Filter: (block <= '11713661'::bigint)
                      Heap Blocks: exact=47562
                      Buffers: shared hit=255908
                      ->  Bitmap Index Scan on idx_address_utxo_owner_addr  (cost=0.00..5211.00 rows=407827 width=0) (actual time=55.382..55.383 rows=381848 loops=1)
                            Index Cond: ((owner_addr)::text = 'addr1v804tgee0m3ww7z93zh64wr9flqh9psdhnxg6cykfudgulg6f633p'::text)
                            Buffers: shared hit=503
                ->  Index Only Scan using idx_input_tx_out_spent_block on tx_input tie1_0  (cost=0.70..2.07 rows=2 width=69) (actual time=0.015..0.015 rows=1 loops=381848)
                      Index Cond: ((tx_hash = (aue1_0.tx_hash)::text) AND (output_index = aue1_0.output_index) AND (spent_at_block <= '11713661'::bigint))
                      Heap Fetches: 11
                      Buffers: shared hit=2041600
        JIT:
          Functions: 63
          Options: Inlining true, Optimization true, Expressions true, Deforming true
          Timing: Generation 7.380 ms, Inlining 340.248 ms, Optimization 682.257 ms, Emission 448.722 ms, Total 1478.607 ms
POST /account/balance
{
    "network_identifier": {
        "blockchain": "cardano",
        "network": "mainnet"
    },
    "account_identifier": {
        "address": "Ae2tdPwUPEYwNguM7TB3dMnZMfZxn1pjGHyGdjaF4mFqZF9L3bj6cdhiH8t",
        "metadata": {}
    }
}
2025-04-09 16:05:21.743 UTC [4345] LOG:  duration: 3120.305 ms  plan:
        Query Text: select aue1_0.output_index,aue1_0.tx_hash,aue1_0.amounts,aue1_0.block,aue1_0.owner_addr,aue1_0.owner_stake_addr from address_utxo aue1_0 where aue1_0.owner_addr=$1 and not exists(select 1 from tx_input tie1_0 where tie1_0.tx_hash=aue1_0.tx_hash and tie1_0.output_index=aue1_0.output_index and tie1_0.spent_at_block<=$2) and aue1_0.block<=$3
        Gather  (cost=6467.33..615053.93 rows=327 width=474) (actual time=1401.061..3118.488 rows=329 loops=1)
          Workers Planned: 6
          Workers Launched: 6
          Buffers: shared hit=2431245 read=1045393
          ->  Nested Loop Anti Join  (cost=6267.33..614837.58 rows=54 width=474) (actual time=1322.627..2840.679 rows=47 loops=7)
                Buffers: shared hit=2431245 read=1045393
                ->  Parallel Bitmap Heap Scan on address_utxo aue1_0  (cost=6266.64..554937.39 rows=80163 width=474) (actual time=424.648..1246.254 rows=82133 loops=7)
                      Recheck Cond: ((owner_addr)::text = 'Ae2tdPwUPEYwNguM7TB3dMnZMfZxn1pjGHyGdjaF4mFqZF9L3bj6cdhiH8t'::text)
                      Filter: (block <= '11713661'::bigint)
                      Heap Blocks: exact=92383
                      Buffers: shared hit=16 read=546628
                      ->  Bitmap Index Scan on idx_address_utxo_owner_addr  (cost=0.00..6146.39 rows=481026 width=0) (actual time=108.797..108.798 rows=575044 loops=1)
                            Index Cond: ((owner_addr)::text = 'Ae2tdPwUPEYwNguM7TB3dMnZMfZxn1pjGHyGdjaF4mFqZF9L3bj6cdhiH8t'::text)
                            Buffers: shared hit=1 read=513
                ->  Index Only Scan using idx_input_tx_out_spent_block on tx_input tie1_0  (cost=0.70..2.06 rows=2 width=69) (actual time=0.019..0.019 rows=1 loops=574929)
                      Index Cond: ((tx_hash = (aue1_0.tx_hash)::text) AND (output_index = aue1_0.output_index) AND (spent_at_block <= '11713661'::bigint))
                      Heap Fetches: 48117
                      Buffers: shared hit=2431229 read=498765
        JIT:
          Functions: 63
          Options: Inlining true, Optimization true, Expressions true, Deforming true
          Timing: Generation 7.344 ms, Inlining 330.803 ms, Optimization 654.584 ms, Emission 430.546 ms, Total 1423.277 ms

@matiwinnetou matiwinnetou changed the base branch from main to develop April 9, 2025 15:51
@matiwinnetou matiwinnetou linked an issue Apr 9, 2025 that may be closed by this pull request
@matiwinnetou matiwinnetou added this to the 1.3.0 milestone Apr 9, 2025
@matiwinnetou matiwinnetou marked this pull request as ready for review April 9, 2025 16:00
@matiwinnetou matiwinnetou modified the milestones: 1.3.0, 1.2.8 Apr 9, 2025
@matiwinnetou matiwinnetou changed the title fix: improve performance on /account/balance for addresses with large transactions count. (quick wins) fix: improve performance on /account/balance for addresses with large transactions count (low effort improvements) Apr 9, 2025
@matiwinnetou matiwinnetou marked this pull request as draft April 9, 2025 19:59
Copy link

@matiwinnetou matiwinnetou marked this pull request as ready for review April 10, 2025 07:33
Base automatically changed from develop to main April 11, 2025 12:29
@matiwinnetou matiwinnetou changed the base branch from main to develop April 11, 2025 13:01
@matiwinnetou
Copy link
Collaborator Author

matiwinnetou commented Apr 16, 2025

We agreed not to merge this but add a special documentation page to Docusaurs related to Rosetta + Postgres performance / scalability tuning for clients using Rosetta. Clients can indices and partial indices based on their individual needs.

@matiwinnetou matiwinnetou removed this from the 1.2.8 milestone Apr 16, 2025
Base automatically changed from develop to main April 29, 2025 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Improvements - /account/balance performance
1 participant