adding json_table function
#1142
Conversation
sql/plan/json_table.go
Outdated
| } | ||
|
|
||
| // NewJSONTable creates a new in memory table from the JSON formatted data, a jsonpath path string, and table spec. | ||
| func NewJSONTable(data []byte, path string, spec *sqlparser.TableSpec, alias sqlparser.TableIdent, schema sql.PrimaryKeySchema) (sql.Node, error) { |
There was a problem hiding this comment.
As in the other PR for vitess, this needs to take an expression for the JSON data, not a []byte
Also sqlparser.* should not make it down to this layer of the code, all that translation should be handled in parse.go
| rowIdx uint64 | ||
| } | ||
|
|
||
| var _ sql.Table = &JSONTable{} |
There was a problem hiding this comment.
You might consider modelling this as a sql.TableFunction instead
There was a problem hiding this comment.
I did originally, but I had trouble implementing the NewInstance method
enginetest/evaluation.go
Outdated
| case float32: | ||
| vw = float64(x) | ||
| case float64: | ||
| vw = float64(float32(x)) |
There was a problem hiding this comment.
There are rounding errors when using float64
There was a problem hiding this comment.
There really shouldn't be, this is a sign of a bug
Floats should round-trip with no precision loss
Does it come from parsing JSON?
enginetest/memory_engine_test.go
Outdated
| enginetest.TestJoinQueries(t, enginetest.NewMemoryHarness("simple", 1, testNumPartitions, true, nil)) | ||
| } | ||
|
|
||
| // TestJoinQueries runs the canonical test queries against a single threaded index enabled harness. |
| "github.com/dolthub/go-mysql-server/sql" | ||
| ) | ||
|
|
||
| var JSONTableQueryTests = []QueryTest{ |
There was a problem hiding this comment.
Need some tests joining json tables to physical tables
If they don't work, add skipped tests
There was a problem hiding this comment.
Also tests where the expression for json data comes from a column, not a string literal
sql/plan/json_table.go
Outdated
| func (t *JSONTable) RowIter(ctx *sql.Context, row sql.Row) (sql.RowIter, error) { | ||
| // TODO: need to resolve function calls like concat() | ||
| // data must evaluate to JSON string | ||
| data, err := t.dataExpr.Eval(ctx, nil) |
enginetest/evaluation.go
Outdated
| case float32: | ||
| vw = float64(x) | ||
| case float64: | ||
| vw = float64(float32(x)) |
There was a problem hiding this comment.
There really shouldn't be, this is a sign of a bug
Floats should round-trip with no precision loss
Does it come from parsing JSON?
sql/plan/json_table.go
Outdated
|
|
||
| // RowIter implements the sql.Node interface | ||
| func (t *JSONTable) RowIter(ctx *sql.Context, row sql.Row) (sql.RowIter, error) { | ||
| // TODO: need to resolve function calls like concat() |
sql/plan/json_table.go
Outdated
| if err != nil { | ||
| return nil, err | ||
| } | ||
| strData, ok := data.(string) |
There was a problem hiding this comment.
Don't cast to a string like this, use Type.Convert()
sql/plan/json_table.go
Outdated
| } | ||
|
|
||
| // Fill in table with data | ||
| for colIdx, p := range t.colPaths { |
There was a problem hiding this comment.
Don't do any of this iteration in RowIter, and don't store it all in memory
Do it one row at a time in iter.Next()
|
|
||
| var JSONTableQueryTests = []QueryTest{ | ||
| { | ||
| Query: "SELECT * FROM JSON_TABLE(NULL,\"$[*]\" COLUMNS(x int path \"$.a\")) as t;", |
There was a problem hiding this comment.
For all these tests, use ' instead of " for your quote delimiter (except where required because it's a json string).
Consider using backtick quoted golang strings so you don't need to escape " in json strings either.
2 participants
Adds some support for
json_tablefunction, which generates a table given a JSON string and column definitions.Fix for: dolthub/dolt#2163
TODO:
NESTEDFOR ORDINALITYON EMPTYON ERROR