Fix broken public interfaces #1761
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Henrik Johansson <henrik@scylladb.com> Co-authored-by: Michał Matczuk <michal@scylladb.com>
This allows tokenAwareHostPolicy return token from ExecutableQuery. Co-authored-by: Henrik Johansson <henrik@scylladb.com> Co-authored-by: Michał Matczuk <michal@scylladb.com>
The hostConnPool logic around conns slice is moved to defaultConnPicker. Co-authored-by: Henrik Johansson <henrik@scylladb.com> Co-authored-by: Michał Matczuk <michal@scylladb.com>
This is an extension to tokenAwareHostPolicy supported by the Scylla 2.3 and onwards. It allows driver to select a connection to shard on a host based on the token. The protocol extension spec is available at [1]. [1] https://github.com/scylladb/scylla/blob/master/docs/protocol-extensions.md Co-authored-by: Henrik Johansson <henrik@scylladb.com> Co-authored-by: Michał Matczuk <michal@scylladb.com>
Signed-off-by: Michał Matczuk <michal@scylladb.com>
As a fork serving Scylla users, we should not assume they are using Cassandra.
A list of excess connections is maintained to allow for lazy removal of. excess connections. Keeping excess connections open helps reaching equilibrium faster since the likelihood of hitting the same shard decreases with the number of connections to the shard. The excess connections list is currently capped to 10 times the number of shards. This magic number has no backing statistics but sounds reasonable.
When the pool is not yet fully materialized we give the caller a random connection to allow the session to start working.
The driver silently replaced the current token with the closest one from the token ring that it could find. This worked well enough to route the request to the correct host but to pick the right shard we need the actual token for the qiven query.
`tokenAwareHostPolicy` always selects a primary replica of a token. When using `tokenAwareHostPolicy` backed by `dcAwareRR`, for tokens with primary replica in a different DC, this results in selecting a RANDOM host in a given DC. This bug was previously fixed in commit scylladb/gocql@39cb625 and apache/cassandra-gocql-driver@ec47935, but was reintroduced in the scylla branch by commit scylladb/gocql@1469a13. The test for token aware policy is updated so that the regression does not happen again.
This option allows to fallback to nodes based by token in
remote DCs before falling back to other nodes in local DC.
This is useful in particular when used with
{'class': 'NetworkTopologyStrategy', 'a': '1', 'b': '1', 'c': '1'}
This way more code is covered by tests. The tests now use stretch/testify as dependency, which should be OK as it was an indirect dependency before.
This test reproduces the race condition.
This commit fixes the next error: ``` type *tokenAwareHostPolicy has no field or method keyspaces ``` which was introduced with 47e1472 after merging scylladb#22 Commit 9867ae9 removed field used in the test and later change was not reflected during ongoing merges.
In case the tuple value has never been written the unmarshalling used to panic. This is now handled by inspecting the data from the database and if it is missing (nil) it will set the tuple value to the corresponding default value. Fixes: apache#1343
Sync with upstream
added training section with links to Scylla University
scylla: integration tests version 1 Authentication tests still remain and some tests have been excluded for various reasons.
Map collections with blob values weren't marshalled into
map[string]interface{}.
This patch allows to store []byte in such map.
Retract tags taken by mistake from kiwicom/gocql
Without this if during restart the resources assigned to node change, the driver does not update this. That could lead to outdated information about threads/shards.
Mark node as down if unable to dial control conn
Fix aggregate bug
In order for Scylla to send the tablet info, the driver must tell the database during connection handshake that it is able to interpret it. It was implemented similarly to the already-present LWT optimisation extension negotiation.
Add mechanism to parse system.tablets periodically. In TokenAwareHostPolicy check if keyspace uses tablets if so try to use them to find replicas. Make shard awareness work when using tablets.
tablet_integration_test.go: Integration tests checking if TokenAwareHostPolicy and shard awareness works correctly when using tablets, and if adding new table changes tablets table. tablet_test.go: Unit tests checking if searching in tablets list works correctly.
Introduce support for tablets
…ablets Update CI to use new way of initializing keyspace with tablets
Previously the check was always based on p.conns[0], but there is a posibility that it is nil and it should be checked. The check should be based on not nil connection This commit introduces searching for first not nil connection and checking tablet-awareness based on this connection.
Fix checking if tablets should be used in scyllaConnPicker Pick
…o a ScyllaDB cluster Fixes: scylladb#170 Signed-off-by: Yaniv Kaul <mykaul@gmail.com>
Do not try to query peers_v2 table upon connection against a ScyllaDB cluster
The java driver has the feature to automatically avoid slow replicas by doing simple heuristics. This is one of the key feature to have a stable latency. This commit adds additional field in tokenAwareHostPolicy to control if the feature is enabled and what is the maximum in flight threshold. If feature is enabled driver sorts the replicas to first try those with less than specified maximum in flight connections. Fixes: apache#154
Send heartbeat OPTIONS message less frequent and enable keep alive
…ions When LWT optimization was implemented in gocql (scylladb#49), all `newFramer` calls were replaced with `newFramerWithExts` calls. Previously, during the upstream merging the code using `newFramer` was added (scylladb#109) and it was forgotten to replace it with `newFramerWithExts`. That way, the `flagLWT` field was set to `0` by default, causing the driver to consider every query executed as a LWT query. The issue was not immediately noticed because the only difference in behavior occurs when we want to use `ShuffleReplicas()` in `TokenAwareHostPolicy`. This commit fixes the issue by replacing `newFramer` with `newFramerWithExts`. Fixes: apache#174
Use `newFramerWithExts` instead of `newFramer` to utilize protocol extensions
Add LOAD_BALANCING_POLICY_SLOW_AVOIDANCE funtionality
currently private interface `token` directly or indirectly present in lot's of interfaces and therefore makes them unimplementable: `SelectedHost`, `ExecutableQuery`, `HostSelectionPolicy`, `NextHost` To fix it we need to make all parts of interfaces be public
|
Wrong repo |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently private interface
tokendirectly or indirectly present in lot's of interfaces and therefore makes them unimplementable:SelectedHost,ExecutableQuery,HostSelectionPolicy,NextHostTo fix it we need to make all parts of interfaces be public