chore: promote algorithm W to master #17900
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…eci config (#16883) Also, update to latest algo-w flux
The repl no longer takes in a querier and it will run everything locally. The spec interface will now not be used and will be removed from the http endpoint at some point.
This adds support for using pkg-config to build libflux inside of the flux dependency. The build can occur by either installing `pkg-config` into your path or the `env` script can be used to invoke it from the go modules.
This updates the repl to support the new influxdb source and use it by default in the repl. It will automatically set some default variables for the influxdb source to make it easier to use the cli. In particular, it will set the default organization, token, and the host. The organization gets set to the one specified in the repl command and the token gets filled in with the user installed one. The host defaults to localhost but will change to whichever one was specified on the cli. In addition, this will replace the http client with one that sets insecure skip verify if the `--skip-verify` flag is used.
The language service abstracts away the parse source which breaks the dependency without moving any of the code.
The tasks subsystem will now use the flux language service to parse and evaluate flux instead of directly interacting with the parser or runtime. This helps break the dependency on the libflux parser for the base influxdb package. This includes the task notification packages which were changed at the same time.
…7109) This removes the storage dependency on libflux by moving the interfaces it implements to the `query` package so it can reference the definitions rather than the package with the implementation and the registration with the runtime. This breaks the dependency where a storage package depends on a flux runtime package.
This removes the spec and updates the lang package usage to make use of passing in the runtime as a parameter. It removes all direct dependendencies on the flux runtime from the http package.
In the past, the default was only being set if a service config wasn't provided. But if a service config was provided and gave a zero value, it would not fill in the default value. This changes the code so that it will always set the default value if the session length is set to zero.
…-w (#17295) Algorithm W will return a semantic graph where every function block always uses a block and a return statement. This is in contrast to the Go code which would have the semantic graph be an expression or a block. The push down code would not introspect blocks which meant that any function expression produced by algorithm w would never be pushed down. This fixes it so the code will now extract the semantic expression from inside of a block if there is exactly one statement and the statement is a return statement. Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
The `buckets()` and `v1.databases()` functions have been updated to support their remote counterparts that were added to flux. These functions now do the same thing as the `from()` call where they will default to the current organization when run against the server and will use the remote versions from the repl.
chore: update flux to latest revision
Added a (disabled and feature-flagged) planner rule that matches:
ReadRange -> window -> { min, max, mean, count, sum }
The rule requires:
* the pushDownWindowAggregate{Count,Rest} feature flags enabled
* having WindowAggregateCapability
(which StorageReader does not currently have)
* use of "_value" columns only
* window.period == window.every
* window.every.months == 0
* window.every is positive
* window.offset == 0
* standard time columns
* createEmpty is false
chore: update flux to latest revision
chore: update flux to latest revision
The tags cache was not thread safe when called from multiple goroutines at the same time. It was intended that it would be, but the locking was done incorrectly and in too complicated a way. There was an assumption that the LRU would only be updated from a single thread which wasn't true at all. The tags cache has now been updated to include some test cases that test for race conditions and data validity. The tags cache itself has been changed to follow a simpler algorithm. 1. Obtain a read lock. 2. Check if the cached array can be used. 3. Release the read lock. 4. If the above was unusable or did not exist, create an array for the tag. 5. Obtain a write lock. 6. Check if the cached array should be replaced and replace if needed. 7. Move the entry to the front of the LRU. 8. Release the write lock. This simpler algorithm should ensure that this code is correct and that creating the array is still done outside of the lock since creating the array is the most expensive operation of the ones above.
chore: merge master into algo-w branch
…ions (#18086) The row functions were refactored to remove state from the dynamic function. This updates the code that invoked row functions.
…#18112) The e2e test driver in influxdb runs the tests twice to get past the fact that there is no way to force order between the write to storage and the read back. When the json.Marshal call became mandatory it was added to the first run, but not the second.
…18116) The column reader passed to `flux.Table.Do` is automatically released. The function passed to the column reader should never release it manually. This causes a double release which causes the table to be erroneously freed when it might be referenced by another transformation. In particular, this affected the following: tables |> yield() |> to() This is because this would produce a buffered table with two references and pass it to both `yield()` and `to()` because `yield()` is a pseudo-node that doesn't really exist. The real graph looks more like: tables |> yield() tables |> to() The `yield()` would double release which would release the `to()` transformation's copy of the column readers. The `to()` method would then be invoked with an invalid column reader.
Added a (disabled) planner rule that matches:
ReadGroupPhys -> { count }
It uses the same physical spec node for group to implement the aggregate. The
rule requires:
* the pushDownGroupAggregateCount feature flag enabled
* no existing aggregate present in the ReadGroup
* use of the "_value" column only
chore: merge master into algo-w branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.