add url function

[lmangani]

Basic macro for url function reading all the supported formats.

Parameters

• url
• format (JSON, CSV, PARQUET, TEXT, BLOB)

Usage

SELECT * FROM url('https://urleng.com/test', 'JSON')

Credits

Thanks @carlopi for the TABLE MACRO solution!

[carlopi]

Cool!

I guess you can make it slightly more robust like:

CREATE OR REPLACE MACRO url(url, format) AS TABLE (
    WITH "internal_JSON" as (SELECT * FROM read_json_auto(url))
        , "internal_CSV" as (SELECT * FROM read_csv(url))
      --- ....
         convert_format as (CASE WHEN format ILIKE 'json' THEN 'internal_JSON'
                             WHEN formal ILIKE 'csv' THEN 'internal_CSV'
                             ------ ....
                             ELSE error('Unsupported parameter, try CSV or JSON') END )
    FROM query_table(convert_format)
);
FROM url('some.csv', 'asdfasdfasdf');

Invalid Input Error: Unsupported parameter, try CSV or JSON

(error is another obscure but quite cool feature, that I am not sure if here is working as intended)

[lmangani]

Thanks for sharing this pattern: I had no idea the error feature existed this is really really cool!
Since this is an experimental extension with no critical mass I guess we can take the risk and adopt obscure tricks ;)

[lmangani]

@carlopi I actually can't make this work with the above example no matter how simple I keep it. Could you share a minimalistic working version? Or does this require 1.1.1?

NOTE: this query drives the Motherduck AI autofix feature nuts :)