Skip to content

Conversation

@Centril
Copy link
Contributor

@Centril Centril commented Sep 2, 2025

Description of Changes

Adds support for truncation in the datastore.
The commitlog format already supported it.
A table is considered truncated in a transaction when there are deletes and when the row count post-deletes is 0.

API and ABI breaking changes

None

Expected complexity level and risk

3?

Testing

Some existing datastore tests have been amended to check whether TxData contain truncation or not.

@Centril Centril changed the base branch from master to centril/clear-table-and-fix-drop-table September 2, 2025 07:51
@bfops bfops added the release-any To be landed in any release window label Sep 3, 2025
@Centril Centril force-pushed the centril/clear-table-and-fix-drop-table branch from 08b70cf to 813f34c Compare September 8, 2025 12:15
Copy link
Contributor

@Shubham8287 Shubham8287 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need to reproduce the bug mentioned in comment, I can help but reasoning seems straight-forward.

@Shubham8287
Copy link
Contributor

I am also getting this error while trying to replay from commitlog after module update.

2025-09-17T09:22:34.516339Z ERROR try_init: crates/core/src/host/module_host.rs:396: failed to create table message

Caused by:
    0: DatastoreError: IndexError: Unique constraint violation 'st_table_table_name_idx_btree' in table 'st_table': column(s): '["table_name"]' value: "message"
    1: IndexError: Unique constraint violation 'st_table_table_name_idx_btree' in table 'st_table': column(s): '["table_name"]' value: "message"
    2: Unique constraint violation 'st_table_table_name_idx_btree' in table 'st_table': column(s): '["table_name"]' value: "message"    

With following repo steps:

  1. publish quickstart module
  2. Make any change in module to change its ModuleDef, ex- remove or add a reducer.
  3. republish module and restart the server.
  4. make a sql call to actually load the module after restart.

Not reproducible in master, I also tried with rebasing the PR on master.

# Description of Changes
Based on,and to fix mentioned issues on
#3215 i.e -
 1.  Mark table for truncation after anaylyzing `TxData::inserts` too.
2. `visit_truncates` can not call `schema_for_tables` for dropped tables
as schema info has been deleted from `st_*` tables. Hence, added a field
in `ReplayVisitor` to cache table names.
 
This PR decouples truncation info from delete entries, since deletion
alone can’t reliably tell about truncation without extra mutations
later.

Can be reviewed commit wise.
# Testing
Testing using "Add columns" PR.
Copy link
Contributor

@Shubham8287 Shubham8287 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged https://github.com/clockworklabs/SpacetimeDB/pull/3261/files which fixes the previously mentiond issues.

@Shubham8287 Shubham8287 changed the base branch from centril/clear-table-and-fix-drop-table to master September 23, 2025 12:19
@Shubham8287 Shubham8287 added this pull request to the merge queue Sep 23, 2025
Merged via the queue into master with commit ddbebbc Sep 23, 2025
24 of 26 checks passed
@Centril Centril deleted the centril/truncate branch September 23, 2025 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-any To be landed in any release window

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants