Skip to content

Latest commit

 

History

History
43 lines (35 loc) · 3.69 KB

query.md

File metadata and controls

43 lines (35 loc) · 3.69 KB

##Query Execution Flow

This document describes the flow of execution of a query engine request.

####Scan Request

Description

  1. Index Coordinator will periodically notify the Indexer about the latest changes in index toplogy. This enables Indexer to maintain a local copy of latest index topologies.
  2. Index Client(query catalog implementation which resides on query server) receives index scan request from the Query Server Component.
  3. Index Client will choose a Indexer node to send this request to. Index Client will have a list of available indexer nodes.
  4. The Index Client will interpret the Consistency/Stability option specified by Query and send the request to the server.

Consistency Options

  • Any Consistency : The indexer would return the most current data available at the moment.
  • Query Consistency : The Query server would specify the minimum acceptable timestamp to it. The index needs to be at or ahead of this to respond. The timestamp may be sparse, in the sense that sequence numbers may be specified for only some vbuckets, but not others. This option allows the query to implement RYOW semantics.
  • Session Consistency : The index client would query the latest timestamp from each KV node. It will ensure that the scan result is at least as recent as the KV timestamp. In other words, this option ensures the query result has seen all mutations globally up to the point when the request was originated.

Stability Options

  • No Stability : The indexer does not ensure any stability of data within each scan. This is the only stability option for consistency-Any.
  • Scan Stability : The indexer would ensure stability of data within each scan.
  • Query Stability : The indexer would ensure stability of data between multiple scans within a query.
  1. Scan Request is sent to the Indexer identified in Step 2 along with the consistency/stability options.
  2. The Indexer receiving the Scan request will become the scan co-ordinator. It checks the local topology information to decide which Indexer nodes will participate in this scan.
  • If index is not found, scan coordinator will return an error "INDEX_NOT_FOUND"
  • If index exists but is in rollback mode, scan coordinator will return an error "INDEX_IN_ROLLBACK"
  1. Based on the consistency option, the indexer will choose a suitable Stability Timestamp. If none is available, it will wait for such Stability Timestamp to become available.
  • Choose latest Stability Timestamp as the Scan Timestamp for the Scan (Any Consistency + Scan/Query Stability).
  • Choose a nil Scan Timestamp for scanning the tip(Any Consistency + No Stability).
  1. Scan coordinator will send all participating indexers(identified in Step 6) the Scan request with Scan Timestamp.
  2. Each local indexer node would use its topology metadata for finding the slices that are required for this scan.
  • If the slices is not active or cannot be found, the local indexer should return an error to the scan coordinator.
  1. Local indexer will:
  • Wait for its local stability timestamp to get past Scan Timestamp(Session Consistency) and run the scan.
  • Run scan against the snapshot matching Scan Timestamp(Any Consistency + Scan/Query Stability).
  • Run the scan on the tip(Any Consistency + No Stability).
    If snapshot is deleted, then the indexer will return an error “SNAPSHOT_TOO_OLD”
  1. Scan results are returned to Scan co-ordinator.
  2. Results are consolidated/aggregated as required.
  3. Results are returned to index client. Local indexer may start streaming the results to index client before all scans are finished for efficiency.