Update 0.5 Reference Docs #43
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
henneberger
reviewed
Aug 20, 2024
| The `primary-key` specifies the column or list of columns that uniquely identifies a single record in the table. Note, that when the table is a changelog or CDC stream for an entity table, the primary key should uniquely identify each record in the stream and not the underlying table. For example, if you consume a CDC stream for a `Customer` entity table with primary key `customerid` the primary key for the resulting CDC stream should include the timestamp of the change, e.g. `[customerid, lastUpdated]`. | ||
|
|
||
| The `timestamp` field specifies the (single) timestamp column for a source stream which has the event time of a single stream record. `watermark-millis` defines the number of milliseconds that events/records can arrive late for consistent processing. Set this to `1` if events are perfectly ordered in time and to `0` if the timestamp is monotonically increasing (i.e. it's perfectly ordered and no two events have the same timestamp). <br /> | ||
| Alternatively, you can also use processing time for event processing by removing the `watermark-millis` field and adding the processing time as metadata (see below), which means using the system clock of the machine processing the data and not the timestamp of the record. We highly recommend you use event time and not processing time for consistent, reproducible results. <br /> |
henneberger
reviewed
Aug 20, 2024
docs/reference/sqrl/connectors.md
Outdated
|
|
||
| ### Connector Configuration | ||
|
|
||
| The connector configuration specifies how the stream engine connects to the source or sink and how it reads or writes the data. The connector configuration is specific to the configured stream processing engine that DataSQRL compiles to and the section of the configuration is named after the engine. In the example above, the connector configuration is for the `flink` engine. |
There was a problem hiding this comment.
really needs a comma after the 'to', hard to read when it ends with a preposition.
The connector configuration is specific to the configured stream processing engine that DataSQRL compiles to, and the section of the configuration is named after the engine.
henneberger
approved these changes
Aug 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.