Skip to content

Conversation

@Centril
Copy link
Contributor

@Centril Centril commented Jan 17, 2025

Description of Changes

Fixes #2021.

Does 2 things:

  1. changes fn {insert,update}_mut_tx to traits.rs to return whether the table was a scheduler table.
  2. adds Table::is_scheduler so datastore_{insert, update}_bsatn don't have to fetch the schema.

Perf before:

for i in {1..5}; do spacetime call basics update_positions_by_collect -s local; done

2025-01-21T04:14:53.970948Z  INFO: : Timing span "update_positions_by_collect": 664.876291ms
2025-01-21T04:14:55.258220Z  INFO: : Timing span "update_positions_by_collect": 651.551835ms
2025-01-21T04:14:56.600896Z  INFO: : Timing span "update_positions_by_collect": 667.1316ms
2025-01-21T04:14:57.722544Z  INFO: : Timing span "update_positions_by_collect": 638.108509ms
2025-01-21T04:14:58.980541Z  INFO: : Timing span "update_positions_by_collect": 642.46669ms

Perf after:

for i in {1..5}; do spacetime call basics update_positions_by_collect -s local; done

2025-01-22T15:21:00.033248Z  INFO: : Timing span "update_positions_by_collect": 663.533985ms
2025-01-22T15:21:01.290030Z  INFO: : Timing span "update_positions_by_collect": 617.702017ms
2025-01-22T15:21:02.417180Z  INFO: : Timing span "update_positions_by_collect": 622.391745ms
2025-01-22T15:21:03.643441Z  INFO: : Timing span "update_positions_by_collect": 618.93754ms
2025-01-22T15:21:04.785486Z  INFO: : Timing span "update_positions_by_collect": 626.819507ms

This means we go from an average 653 ms to 630 ms, which is about an average 23 ms speedup.
This was the expected speedup based on % of samples in the update ABI PR flamegraph.

API and ABI breaking changes

None

Expected complexity level and risk

1, fairly simple and contained change.

Testing

A test of the caching behavior is added in datastore.rs: fn test_scheduled_table_insert_and_update.

@Centril Centril force-pushed the centril/update-abi branch 2 times, most recently from bc2a869 to f388240 Compare January 22, 2025 12:00
@Centril Centril force-pushed the centril/memo-is-scheduler branch 3 times, most recently from 52dc83b to 91fe2fb Compare January 27, 2025 16:04
@Centril Centril force-pushed the centril/update-abi branch from f388240 to a1aeb6c Compare January 27, 2025 16:10
@Centril Centril force-pushed the centril/memo-is-scheduler branch from 91fe2fb to 698b526 Compare January 27, 2025 16:34
@Centril Centril marked this pull request as ready for review January 27, 2025 16:37
@cloutiertyler
Copy link
Contributor

Why are we returning this per insert when we can know whether or not the table we're inserting into is a scheduled table a piori?

@Centril
Copy link
Contributor Author

Centril commented Jan 27, 2025

Why are we returning this per insert when we can know whether or not the table we're inserting into is a scheduled table a piori?

Semantically, we can check after or before. The previous code did check after, but it is unfortunately slower, as it incurs another table_scheduled_id_and_at -> schema_for_table call, which we can avoid, by sending the information in insert / update instead, essentially for free, since we're fetching the Table there anyways. So the answer boils down to: "semantically, this is not pretty, but it is more efficient".

Copy link
Contributor

@cloutiertyler cloutiertyler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, I approve the changes to traits.rs. Please get a full review from the primary reviewer.

Copy link
Contributor

@gefjon gefjon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Why is InsertFlags a distinct type from UpdateFlags?
  • Now that we've got insert and update returning a 3-tuple, is it time to switch it to a struct with named fields?

Base automatically changed from centril/update-abi to master January 27, 2025 20:11
Copy link
Contributor

@gefjon gefjon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, pending TODO comments to the effect of my earlier questions.

@bfops bfops added the release-any To be landed in any release window label Jan 28, 2025
@Centril Centril force-pushed the centril/memo-is-scheduler branch from 698b526 to 209e4ae Compare January 28, 2025 19:29
@Centril Centril force-pushed the centril/memo-is-scheduler branch from 209e4ae to 5f6b3ae Compare January 28, 2025 19:31
@Centril Centril enabled auto-merge January 28, 2025 19:31
@Centril Centril added this pull request to the merge queue Jan 28, 2025
Merged via the queue into master with commit 5b5a0a6 Jan 28, 2025
13 checks passed
@Centril Centril deleted the centril/memo-is-scheduler branch January 28, 2025 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-any To be landed in any release window

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: have insert return whether it is a scheduler table or not

5 participants