Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize pilorama #1431

Merged
merged 3 commits into from
May 30, 2022
Merged

Optimize pilorama #1431

merged 3 commits into from
May 30, 2022

Conversation

fyrchik
Copy link
Contributor

@fyrchik fyrchik commented May 25, 2022

  1. Update -> Batch.
  2. Build index on FileName attribute.

@KirillovDenis KirillovDenis mentioned this pull request May 25, 2022
Closed
carpawell
carpawell reviewed May 27, 2022
View reviewed changes
Copy link
Member

@carpawell carpawell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you, please, fix the commit messages?

@fyrchik
[nspcc-dev#1431] engine: Add benchmark for Select vs TreeGetByPath
54e607c 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik
[nspcc-dev#1431] pilorama: Cache attributes in the index
5c9ed2b 
Currently to find a node by path we iterate over all the children on
each level. This is far from optimal and scales badly with the number of
nodes on a single level. Thus we introduce "indexed attributes" for
which an additional information is stored and which can be use in
`*ByPath` operations. Currently this set only includes `FileName`
attribute but this may change in future.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik fyrchik mentioned this pull request May 27, 2022
Open
@fyrchik
[nspcc-dev#1431] pilorama: Use Batch for write transactions
e3a9a7d 
Helps a lot in case of concurrent request flow.

```
name                      old time/op    new time/op    delta
ApplySequential/bbolt-8     78.0µs ± 9%    59.8µs ± 4%  -23.39%  (p=0.000 n=10+9)
ApplyReorderLast/bbolt-8     143µs ± 5%     113µs ±15%  -21.06%  (p=0.000 n=10+10)

name                      old alloc/op   new alloc/op   delta
ApplySequential/bbolt-8     56.9kB ± 8%    28.9kB ± 3%  -49.22%  (p=0.000 n=10+10)
ApplyReorderLast/bbolt-8    87.3kB ± 3%    40.9kB ±10%  -53.16%  (p=0.000 n=10+10)

name                      old allocs/op  new allocs/op  delta
ApplySequential/bbolt-8        224 ±11%       262 ± 5%  +16.93%  (p=0.000 n=9+10)
ApplyReorderLast/bbolt-8       518 ± 4%       674 ±11%  +30.09%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
@fyrchik fyrchik mentioned this pull request May 28, 2022
Merged
@fyrchik fyrchik merged commit c70c73c into nspcc-dev:tree-service May 30, 2022
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1431] engine: Add benchmark for Select vs TreeGetByPath
43ed4ef 
Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1431] pilorama: Cache attributes in the index
6f0ea8f 
Currently to find a node by path we iterate over all the children on
each level. This is far from optimal and scales badly with the number of
nodes on a single level. Thus we introduce "indexed attributes" for
which an additional information is stored and which can be use in
`*ByPath` operations. Currently this set only includes `FileName`
attribute but this may change in future.

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
aprasolova pushed a commit to aprasolova/neofs-node that referenced this pull request Oct 19, 2022
@fyrchik @aprasolova
[nspcc-dev#1431] pilorama: Use Batch for write transactions
c62c85b 
Helps a lot in case of concurrent request flow.

```
name                      old time/op    new time/op    delta
ApplySequential/bbolt-8     78.0µs ± 9%    59.8µs ± 4%  -23.39%  (p=0.000 n=10+9)
ApplyReorderLast/bbolt-8     143µs ± 5%     113µs ±15%  -21.06%  (p=0.000 n=10+10)

name                      old alloc/op   new alloc/op   delta
ApplySequential/bbolt-8     56.9kB ± 8%    28.9kB ± 3%  -49.22%  (p=0.000 n=10+10)
ApplyReorderLast/bbolt-8    87.3kB ± 3%    40.9kB ±10%  -53.16%  (p=0.000 n=10+10)

name                      old allocs/op  new allocs/op  delta
ApplySequential/bbolt-8        224 ±11%       262 ± 5%  +16.93%  (p=0.000 n=9+10)
ApplyReorderLast/bbolt-8       518 ± 4%       674 ±11%  +30.09%  (p=0.000 n=10+10)
```

Signed-off-by: Evgenii Stratonikov <evgeniy@nspcc.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@carpawell carpawell carpawell approved these changes

@alexvanin alexvanin Awaiting requested review from alexvanin

@cthulhu-rider cthulhu-rider Awaiting requested review from cthulhu-rider

@KirillovDenis KirillovDenis Awaiting requested review from KirillovDenis

@realloc realloc Awaiting requested review from realloc

Assignees

@fyrchik fyrchik

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fyrchik @carpawell