-
Notifications
You must be signed in to change notification settings - Fork 320
docs(influxdb3): document duplicate point write ordering and workarounds #6584
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Expand duplicate points documentation for Cloud Dedicated and Clustered to address non-deterministic write ordering when duplicate points are flushed together. Changes: - Add warning callout explaining duplicate point overwrites are non-deterministic - Add recommended patterns section with append-only approaches - Add SQL and InfluxQL query examples for getting latest state - Add anti-patterns section with common mistakes to avoid - Add retention guidance for last-value tables (Cloud Dedicated only) - Add performance considerations for append-only patterns - Add cross-links from schema-design and optimize-writes pages closes influxdata/DAR#560
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
|
@helenosheaa Specifically, can you help answer Questions for Engineering in the description? |
Yes it would also apply to Serverless
It would be the same yes |
|
This is related for Enterprise v3: InfluxDB v3 Enterprise uses the same underlying write semantics, so the non-determinism with duplicate points in the same batch can occur there as well. The difference is that Enterprise gives you more control over how long data stays in the “gen1” tier before being compacted/flushed via the gen1-duration setting: https://docs.influxdata.com/influxdb3/enterprise/reference/config-options/#gen1-duration |
Thanks @will-influxdata! I'll add it for Enterprise in a separate PR. |
Summary
Expands duplicate points documentation for Cloud Dedicated and Clustered to address non-deterministic write ordering when duplicate points are flushed together.
Changes
Questions for Engineering
Should the following also apply to Clustered?
Should this documentation also apply to Cloud Serverless?
The current changes are scoped to Cloud Dedicated and Clustered only (matching the original documentation scope). However, the
optimize-writes.mdpage for Cloud Serverless already contains a "Write ordering for duplicate points" callout with similar content.Can Engineering confirm whether Cloud Serverless has the same ingestion/flushing behavior that causes non-deterministic ordering when duplicate points are flushed together? If so, we should extend this documentation to include Cloud Serverless as well.
Test plan
closes influxdata/DAR#560