Skip to content

Conversation

@jycor
Copy link
Contributor

@jycor jycor commented Jan 19, 2023

Changes:

  • Add feature flag to enabled spatial indexes: DOLT_ENABLE_SPATIAL_INDEX
  • Changes to flatbuffer stuff to read/write spatial indexes
  • First pass at naive and unoptimized implementation of Spatial Indexes
  • Tests for Spatial Indexes, Spatial Index Plans, and their Prepared counterparts (some are skipped)
  • Contains some fixes for typeinfo not retaining SRID information for multi-geometries
  • Changes to fix some diff table prepared issues

Companion PR: dolthub/go-mysql-server#1558
Note: prepared test that I changed here had a column anmed commit, which got confused with to_commit in the diff #5188

TODOs

  • Proper benchmarks
  • Potential Optimizations:
    • Level Pruning
    • Z-BBox skipping
    • Subdividing queries
    • Make Spatial Indexes Covering
    • Multi-Threading Level Searches

@jycor jycor marked this pull request as draft January 20, 2023 19:56
@jycor jycor changed the title implementing z-order encoding for points implement spatial indexes Jan 23, 2023
Copy link
Contributor

@andy-wm-arthur andy-wm-arthur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good, some remaining work to clean this up, then ship it

Tags []uint64 `noms:"tags" json:"tags"`
Comment string `noms:"comment" json:"comment"`
Unique bool `noms:"unique" json:"unique"`
Spatial bool `noms:"spatial,omitempty" json:"spatial,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should revert these changes, we don't support spatial indexes in the old format. you should also add some tests to confirm this

Comment on lines +184 to 192
return &multipointType{sqlType.(gmstypes.MultiPointType)}, nil
case gmstypes.MultiLineStringType{}.String():
return &multilinestringType{}, nil
return &multilinestringType{sqlType.(gmstypes.MultiLineStringType)}, nil
case gmstypes.MultiPolygonType{}.String():
return &multipolygonType{}, nil
return &multipolygonType{sqlType.(gmstypes.MultiPolygonType)}, nil
case gmstypes.GeomCollType{}.String():
return &geomcollType{}, nil
return &geomcollType{sqlType.(gmstypes.GeomCollType)}, nil
case gmstypes.GeometryType{}.String():
return &geometryType{sqlGeometryType: sqlType.(gmstypes.GeometryType)}, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this change for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something I should've implemented when I was originally implementing MultiPoint, MultiLineString, MultiPolygon, and GeometryCollection. When defining columns over these types, we're able to specify a default SRID, and sqlType is what remembers the columns user defined SRID.

Example: create table t (p point srid 0)

Having a SRID defined is required for Spatial Indexes to work.

// TODO: the descriptors in the primary key are different
// than the ones in the secondary key; this test assumes
// they're the same
if len(def.PrefixLengths()) > 0 || def.IsSpatial() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implement the logic to validate both of these speacial case indexes, rather than just skipping them. otherwise you're eroding the value of this validation

@jycor jycor merged commit 146a1b9 into main Feb 21, 2023
@Hydrocharged Hydrocharged deleted the james/spatial-index-test branch February 23, 2023 20:19
@Hydrocharged Hydrocharged restored the james/spatial-index-test branch February 23, 2023 20:20
@Hydrocharged Hydrocharged deleted the james/spatial-index-test branch February 23, 2023 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants