the query builder doesn't handle null values for json fields

[haikyuu]

Query looks like this

const query = e.params(
  {raw_data: e.array(e.tuple({title: e.str}))},
  params =>
    e.for(e.array_unpack(params.raw_data), item =>
      e.insert(e.Movie, {
        title: item.title,
        github_scopes: e.cast(e.array(e.str), item.github_scopes)
      })
    )
);
console.log(query.toEdgeQL());
const result = await query.run(client, {
  raw_data: [
    {title: "The Marvels", github_scopes: null},
  ],
});

I'm getting the following error (could use a better message too #275 )

InvalidValueError: cannot extract elements from a scalar
    at RawConnection._parseErrorMessage (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/rawConn.js:371:21)
    at RawConnection._executeFlow (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/rawConn.js:861:34)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async RawConnection.fetch (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/rawConn.js:981:13)
    at async Transaction._runOp (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/transaction.js:94:20)
    at null.<anonymous> (/Users/abdellah/workspace/scrimba/scrimba-edge/dbschema/pg_migrations/users.imba:157:24)
    at async ClientConnectionHolder.transaction (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/client.js:135:26)
    at async Client.transaction (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/client.js:450:20)
    at null.run (/Users/abdellah/workspace/scrimba/scrimba-edge/dbschema/pg_migrations/users.imba:109:8) {
  source: undefined
}

And it happens because of this line github_scopes: e.cast(e.array(e.str), item.github_scopes) in a json unpack insert callback where the value of github_scopes is null

[haikyuu]

When I omit the value from the object delete user.github_scopes I get

InvalidValueError: JSON index 'github_scopes' is out of bounds
    at RawConnection._parseErrorMessage (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/rawConn.js:371:21)
    at RawConnection._executeFlow (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/rawConn.js:861:34)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async RawConnection.fetch (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/rawConn.js:981:13)
    at async Transaction._runOp (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/transaction.js:94:20)
    at null.<anonymous> (/Users/abdellah/workspace/scrimba/scrimba-edge/dbschema/pg_migrations/users.imba:157:24)
    at async ClientConnectionHolder.transaction (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/client.js:135:26)
    at async Client.transaction (/Users/abdellah/workspace/scrimba/scrimba-edge/node_modules/edgedb/dist/client.js:450:20)
    at null.run (/Users/abdellah/workspace/scrimba/scrimba-edge/dbschema/pg_migrations/users.imba:109:8) {
  source: undefined
}

[colinhacks]

FFR, I recommend logging the generated EdgeQL to try to debug stuff like this. Just take your query builder expression and call the .toEdgeQL() method to get the EdgeQL representation as a string. This is also useful for us when debugging.

---

Seems like you've provided the wrong query? There is no JSON field in this query. Your raw_type parameter is of type array<tuple<title: str>>. After unpacking that, you have tuple<title: str>, so github_scopes is naturally not a valid key.

Here's that same query rewritten to use a parameter of type json.

  const query = e.params({raw_data: e.json}, params =>
    e.for(e.json_array_unpack(params.raw_data), item => {
      return e.insert(e.Movie, {
        title: e.cast(e.str, item.title),
        github_scopes: e.cast(e.array(e.str), item.github_scopes),
      });
    })
  );

  const result = await query.run(client, {
    raw_data: JSON.stringify([{title: "The Marvels", github_scopes: [] }]),
  });

Since you're casting item.github_scopes to array<str>, the appropriate value to represent an empty value is []. This isn't a query builder problem, this is due to how EdgeQL works. Here's a much simpler example where you can see this behavior. You can copy/paste this query into the REPL to experiment - run edgedb inside your project directory to open the REPL.

edgedb> WITH
.......   data := <json>$data
....... SELECT {
.......   title := <str>(data["title"]),
.......   github_scopes := <array<str>>(data["github_scopes"])
....... };
Parameter <json>$data: {"title": "Test", "github_scopes": null}
edgedb error: InvalidValueError: cannot extract elements from a scalar
edgedb> WITH
.......   data := <json>$data
....... SELECT {
.......   title := <str>(data["title"]),
.......   github_scopes := <array<str>>(data["github_scopes"])
....... };
Parameter <json>$data: {"title": "Test", "github_scopes": []}
{{title: 'Test', github_scopes: []}}