Skip to content

Commit 502ff2a

Browse files
rustyrussellrustyrussell
committed
plugins/sql: listsqlschemas command to retrieve schemas.
Good for detection of what fields are present. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1 parent f5c9c52 commit 502ff2a

File tree

9 files changed

+334
-0
lines changed

9 files changed

+334
-0
lines changed

‎doc/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ MANPAGES := doc/lightning-cli.1 \
5858
doc/lightning-listpays.7 \
5959
doc/lightning-listpeers.7 \
6060
doc/lightning-listsendpays.7 \
61+
doc/lightning-listsqlschemas.7 \
6162
doc/lightning-makesecret.7 \
6263
doc/lightning-multifundchannel.7 \
6364
doc/lightning-multiwithdraw.7 \

‎doc/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ Core Lightning Documentation
8585
lightning-listpays <lightning-listpays.7.md>
8686
lightning-listpeers <lightning-listpeers.7.md>
8787
lightning-listsendpays <lightning-listsendpays.7.md>
88+
lightning-listsqlschemas <lightning-listsqlschemas.7.md>
8889
lightning-listtransactions <lightning-listtransactions.7.md>
8990
lightning-makesecret <lightning-makesecret.7.md>
9091
lightning-multifundchannel <lightning-multifundchannel.7.md>

‎doc/lightning-listsqlschemas.7.md

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
lightning-listsqlschemas -- Command to example lightning-sql schemas
2+
====================================================================
3+
4+
SYNOPSIS
5+
--------
6+
7+
**listsqlschemas** [*table*]
8+
9+
DESCRIPTION
10+
-----------
11+
12+
This allows you to examine the schemas at runtime; while they are fully
13+
documented for the current release in lightning-sql(7), as fields are
14+
added or deprecated, you can use this command to determine what fields
15+
are present.
16+
17+
If *table* is given, only that table is in the resulting list, otherwise
18+
all tables are listed.
19+
20+
EXAMPLE JSON REQUEST
21+
------------
22+
```json
23+
{
24+
"id": 82,
25+
"method": "listsqlschemas",
26+
"params": {
27+
"table": "offers"
28+
}
29+
}
30+
```
31+
32+
EXAMPLE JSON RESPONSE
33+
-----
34+
```json
35+
{
36+
"schemas": [
37+
{
38+
"tablename": "offers",
39+
"columns": [
40+
{
41+
"name": "offer_id",
42+
"type": "BLOB"
43+
},
44+
{
45+
"name": "active",
46+
"type": "INTEGER"
47+
},
48+
{
49+
"name": "single_use",
50+
"type": "INTEGER"
51+
},
52+
{
53+
"name": "bolt12",
54+
"type": "TEXT"
55+
},
56+
{
57+
"name": "bolt12_unsigned",
58+
"type": "TEXT"
59+
},
60+
{
61+
"name": "used",
62+
"type": "INTEGER"
63+
},
64+
{
65+
"name": "label",
66+
"type": "TEXT"
67+
}
68+
],
69+
"indices": [
70+
[
71+
"offer_id"
72+
]
73+
]
74+
}
75+
]
76+
}
77+
```
78+
79+
RETURN VALUE
80+
------------
81+
82+
[comment]: # (GENERATE-FROM-SCHEMA-START)
83+
On success, an object containing **schemas** is returned. It is an array of objects, where each object contains:
84+
85+
- **tablename** (string): the name of the table
86+
- **columns** (array of objects): the columns, in database order:
87+
- **name** (string): the name the column
88+
- **type** (string): the SQL type of the column (one of "INTEGER", "BLOB", "TEXT", "REAL")
89+
- **indices** (array of arrays, optional): Any index we created to speed lookups:
90+
- The columns for this index:
91+
- The column name
92+
93+
[comment]: # (GENERATE-FROM-SCHEMA-END)
94+
95+
AUTHOR
96+
------
97+
98+
Rusty Russell <<rusty@rustcorp.com.au>> is mainly responsible.
99+
100+
SEE ALSO
101+
--------
102+
103+
lightning-sql(7).
104+
105+
RESOURCES
106+
---------
107+
108+
Main web site: <https://github.com/ElementsProject/lightning>
109+
[comment]: # ( SHA256STAMP:4ce219bf63d1ce5f9e3337c767f3258bb758b5ab474fe2792df4daa78165aded)

‎doc/lightning-sql.7.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ cache `listnodes` and `listchannels`) which then processes the results.
1919
It is, however faster for remote access if the result of the query is
2020
much smaller than the list commands would be.
2121

22+
Note that queries like "SELECT *" are fragile, as columns will
23+
change across releases; see lightning-listsqlschemas(7).
24+
2225
TREATMENT OF TYPES
2326
------------------
2427

‎doc/lightning-sqlsimple.7.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@ array of objects using those names instead of an array of arrays.
1616
The query is build by prepending "SELECT" to the *columns" array, then
1717
appending the *query* string.
1818

19+
Note that using "*" for `columns` is fragile, as results will
20+
change across releases; see lightning-listsqlschemas(7).
21+
1922
EXAMPLE JSON REQUEST
2023
------------
2124
```json

‎doc/schemas/listsqlschemas.request.json

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"type": "object",
4+
"required": [],
5+
"properties": {
6+
"table": {
7+
"type": "string"
8+
}
9+
}
10+
}

‎doc/schemas/listsqlschemas.schema.json

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"type": "object",
4+
"additionalProperties": false,
5+
"required": [
6+
"schemas"
7+
],
8+
"properties": {
9+
"schemas": {
10+
"type": "array",
11+
"items": {
12+
"type": "object",
13+
"additionalProperties": false,
14+
"required": [
15+
"tablename",
16+
"columns"
17+
],
18+
"properties": {
19+
"tablename": {
20+
"type": "string",
21+
"description": "the name of the table"
22+
},
23+
"columns": {
24+
"type": "array",
25+
"description": "the columns, in database order",
26+
"items": {
27+
"type": "object",
28+
"additionalProperties": false,
29+
"required": [
30+
"name",
31+
"type"
32+
],
33+
"properties": {
34+
"name": {
35+
"type": "string",
36+
"description": "the name the column"
37+
},
38+
"type": {
39+
"type": "string",
40+
"enum": [
41+
"INTEGER",
42+
"BLOB",
43+
"TEXT",
44+
"REAL"
45+
],
46+
"description": "the SQL type of the column"
47+
}
48+
}
49+
}
50+
},
51+
"indices": {
52+
"type": "array",
53+
"description": "Any index we created to speed lookups",
54+
"items": {
55+
"type": "array",
56+
"description": "The columns for this index",
57+
"items": {
58+
"type": "string",
59+
"description": "The column name"
60+
}
61+
}
62+
}
63+
}
64+
}
65+
}
66+
}
67+
}

‎plugins/sql.c

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1079,6 +1079,111 @@ static struct command_result *json_sqlsimple(struct command *cmd,
10791079
return do_query(cmd, dbq, query);
10801080
}
10811081

1082+
static struct command_result *param_tablename(struct command *cmd,
1083+
const char *name,
1084+
const char *buffer,
1085+
const jsmntok_t *tok,
1086+
struct table_desc **td)
1087+
{
1088+
*td = strmap_getn(&tablemap, buffer + tok->start,
1089+
tok->end - tok->start);
1090+
if (!*td)
1091+
return command_fail_badparam(cmd, name, buffer, tok,
1092+
"Unknown table");
1093+
return NULL;
1094+
}
1095+
1096+
static void json_add_column(struct json_stream *js,
1097+
const char *dbname,
1098+
const char *sqltypename)
1099+
{
1100+
json_object_start(js, NULL);
1101+
json_add_string(js, "name", dbname);
1102+
json_add_string(js, "type", sqltypename);
1103+
json_object_end(js);
1104+
}
1105+
1106+
static void json_add_columns(struct json_stream *js,
1107+
const struct table_desc *td)
1108+
{
1109+
for (size_t i = 0; i < tal_count(td->columns); i++) {
1110+
if (td->columns[i].sub) {
1111+
if (td->columns[i].sub->is_subobject)
1112+
json_add_columns(js, td->columns[i].sub);
1113+
continue;
1114+
}
1115+
json_add_column(js, td->columns[i].dbname,
1116+
fieldtypemap[td->columns[i].ftype].sqltype);
1117+
}
1118+
}
1119+
1120+
static void json_add_schema(struct json_stream *js,
1121+
const struct table_desc *td)
1122+
{
1123+
bool have_indices;
1124+
1125+
json_object_start(js, NULL);
1126+
json_add_string(js, "tablename", td->name);
1127+
/* This needs to be an array, not a dictionary, since dicts
1128+
* are often treated as unordered, and order is critical! */
1129+
json_array_start(js, "columns");
1130+
if (td->parent) {
1131+
json_add_column(js, "row", "INTEGER");
1132+
json_add_column(js, "arrindex", "INTEGER");
1133+
}
1134+
json_add_columns(js, td);
1135+
json_array_end(js);
1136+
1137+
/* Don't print indices entry unless we have an index! */
1138+
have_indices = false;
1139+
for (size_t i = 0; i < ARRAY_SIZE(indices); i++) {
1140+
if (!streq(indices[i].tablename, td->name))
1141+
continue;
1142+
if (!have_indices) {
1143+
json_array_start(js, "indices");
1144+
have_indices = true;
1145+
}
1146+
json_array_start(js, NULL);
1147+
for (size_t j = 0; j < ARRAY_SIZE(indices[i].fields); j++) {
1148+
if (indices[i].fields[j])
1149+
json_add_string(js, NULL, indices[i].fields[j]);
1150+
}
1151+
json_array_end(js);
1152+
}
1153+
if (have_indices)
1154+
json_array_end(js);
1155+
json_object_end(js);
1156+
}
1157+
1158+
static bool add_one_schema(const char *member, struct table_desc *td,
1159+
struct json_stream *js)
1160+
{
1161+
json_add_schema(js, td);
1162+
return true;
1163+
}
1164+
1165+
static struct command_result *json_listsqlschemas(struct command *cmd,
1166+
const char *buffer,
1167+
const jsmntok_t *params)
1168+
{
1169+
struct table_desc *td;
1170+
struct json_stream *ret;
1171+
1172+
if (!param(cmd, buffer, params,
1173+
p_opt("table", param_tablename, &td),
1174+
NULL))
1175+
return command_param_failed();
1176+
1177+
ret = jsonrpc_stream_success(cmd);
1178+
json_array_start(ret, "schemas");
1179+
if (td)
1180+
json_add_schema(ret, td);
1181+
else
1182+
strmap_iterate(&tablemap, add_one_schema, ret);
1183+
json_array_end(ret);
1184+
return command_finished(cmd, ret);
1185+
}
1186+
10821187
/* Creates sql statements, initializes table */
10831188
static void finish_td(struct plugin *plugin, struct table_desc *td)
10841189
{
@@ -1411,6 +1516,13 @@ static const struct plugin_command commands[] = {
14111516
"This is the greatest plugin command ever!",
14121517
json_sqlsimple,
14131518
},
1519+
{
1520+
"listsqlschemas",
1521+
"misc",
1522+
"Display schemas for internal sql tables, or just {table}",
1523+
"This is the greatest plugin command ever!",
1524+
json_listsqlschemas,
1525+
},
14141526
};
14151527

14161528
static const char *fmt_indexes(const tal_t *ctx, const char *table)

‎tests/test_plugin.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3751,6 +3751,34 @@ def test_sql(node_factory, bitcoind):
37513751
{'name': 'payment_id',
37523752
'type': 'hex'}]}}
37533753

3754+
sqltypemap = {'string': 'TEXT',
3755+
'boolean': 'INTEGER',
3756+
'u8': 'INTEGER',
3757+
'u16': 'INTEGER',
3758+
'u32': 'INTEGER',
3759+
'u64': 'INTEGER',
3760+
'msat': 'INTEGER',
3761+
'hex': 'BLOB',
3762+
'hash': 'BLOB',
3763+
'txid': 'BLOB',
3764+
'pubkey': 'BLOB',
3765+
'secret': 'BLOB',
3766+
'number': 'REAL',
3767+
'short_channel_id': 'TEXT'}
3768+
3769+
# Check schemas match.
3770+
for table, schema in expected_schemas.items():
3771+
res = only_one(l2.rpc.listsqlschemas(table)['schemas'])
3772+
assert res['tablename'] == table
3773+
assert res.get('indices') == schema.get('indices')
3774+
sqlcolumns = [{'name': c['name'], 'type': sqltypemap[c['type']]} for c in schema['columns']]
3775+
assert res['columns'] == sqlcolumns
3776+
3777+
# Make sure we didn't miss any
3778+
assert (sorted([s['tablename'] for s in l1.rpc.listsqlschemas()['schemas']])
3779+
== sorted(expected_schemas.keys()))
3780+
assert len(l1.rpc.listsqlschemas()['schemas']) == len(expected_schemas)
3781+
37543782
# Very rough checks of other list commands (make sure l2 has one of each)
37553783
l2.rpc.offer(1, 'desc')
37563784
l2.rpc.invoice(1, 'label', 'desc')

0 commit comments

Comments
 (0)