Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document reason for datastream timestamp field name #5200

Open
cthtrifork opened this issue Oct 11, 2023 · 6 comments
Open

Document reason for datastream timestamp field name #5200

cthtrifork opened this issue Oct 11, 2023 · 6 comments
Labels
1 - Backlog Issue: The issue is unassigned or assigned but not started Sev3 Medium priority. Content that's missing, driven by dev, PM or the community.

Comments

@cthtrifork
Copy link

cthtrifork commented Oct 11, 2023

Is your feature request related to a problem? Please describe.
I have tried looking at the code and the documentation for datastream. I can not find documentation that describes why a timestamp field is needed on each document.
Rollover is based on index timestamp and each new index is created with a generationId. I can not spot anywhere the timestamp on a document has a technical impact.

Describe the solution you'd like
Update https://opensearch.org/docs/2.10/im-plugin/data-streams/ with a technical reason for the timestamp requiremement on documents.

Describe alternatives you've considered
Make the field optional if it has no functional impact

@cthtrifork cthtrifork added enhancement New feature or request untriaged labels Oct 11, 2023
@noCharger
Copy link

@opensearch-project/admin Please transfer this to https://github.com/opensearch-project/documentation-website. Thanks!

@bbarani bbarani transferred this issue from opensearch-project/OpenSearch Oct 13, 2023
@Naarcha-AWS
Copy link
Collaborator

@noCharger: Is there a technical reason for need on timestamp on each document?

@Naarcha-AWS Naarcha-AWS added 1 - Backlog Issue: The issue is unassigned or assigned but not started Sev3 Medium priority. Content that's missing, driven by dev, PM or the community. and removed enhancement New feature or request untriaged labels Oct 16, 2023
@cthtrifork
Copy link
Author

@noCharger I am permitting myself to bump this, hope it is okay,

@cthtrifork
Copy link
Author

This seems related: #2811

@hdhalter
Copy link
Contributor

HI @bowenlan-amzn, can you please help us update the documentation for this issue?

@bowenlan-amzn
Copy link
Member

Data stream function is for time-series data, so requiring the timestamp field. It's not a functional issue, but just the feature.
@ketanv3 would the data stream expert to help explain more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 - Backlog Issue: The issue is unassigned or assigned but not started Sev3 Medium priority. Content that's missing, driven by dev, PM or the community.
Projects
None yet
Development

No branches or pull requests

5 participants