Error while reading again a parquet file after browser reload

[ericemc3]

What happens?

Executing twice the same request, after reloading the shell page, yields an error.

To Reproduce

in https://shell.duckdb.org/, execute :

FROM 'https://static.data.gouv.fr/resources/communes-2023-format-parquet/20240122-085355/communes2023.parquet' 
SELECT codgeo WHERE epci = '200039865' ;

then reload the browser and execute that same query again.

On windows and with Chrome or Edge, i get:
Invalid Error: TProtocolException: Invalid data

codgeo column, which is also the first of the dataset, seems to be responsible.

With Firefox, no issue.

OS:

Win11

DuckDB Version:

10.0.0

DuckDB Client:

shell wasm1.28.1-dev159.0

Full Name:

eric mauviere

Affiliation:

icem7

Have you tried this on the latest nightly build?

I have tested with a nightly build

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• Yes, I have

[carlopi]

Hi @ericemc3, can you share the result of PRAGMA user_agent on the SQL side?

I am aware of a problem with threads, but on the browser I tested it was NOT enable by default, but your setup is somewhat different.

IFF the result contains wasm_threads, I am aware of the problem and working on a fix. You can specify explicitly the bundle going to https://shell.duckdb.org/?bundle=eh

IFF the result is wasm_eh, this is a new problem, and I will later look at that.

[carlopi]

(and thanks @szarnyasg, this looks like a duckdb/duckdb-wasm specific problem)

[ericemc3]

Hi @carlopi,
thank you for your prompt reaction.
I get
┌─────────────────────────────┐
│ user_agent │
╞═════════════════════════════╡
│ duckdb/v0.10.0(wasm_eh) cpp │

and the same issue with https://shell.duckdb.org/?bundle=eh

Please note that it works fine with Firefox (with the same user_agent displayed). So it looks like a Chrome/Edge specific issue as well.

[dude0001]

The same thing can be duplicated in DuckDB-WASM v1.28.0 (DuckDB v0.9.1). I thought this was an issue in my app and was still troubleshooting as a low priority, so I haven't reported an issue yet. I just saw this pop up so thought I'd share we see this in v1.28.0 as well. I have worked around it for the moment by fetching the files outside of DuckDB doing registerFileBuffer and read_parquet over the registered files instead of URLs of the parquet files.

[timothyhoward]

I've been experiencing this same issue while using Evidence (@evidence-dev/evidence), which uses @duckdb/duckdb-wasm as a dependency. Seems to be affecting Windows, and only Edge/Chrome. Page may load initially or error out, displaying Invalid error: TProtocolException: Invalid data. Turning off caching, or going into Incognito/InPrivate mode, and the query/page works correctly. We're querying Parquet files using the URL, just like this issue.

Affecting 1.28.0 (the Evidence dependency version) and the latest build 1.28.1-dev190.0.

[fboerman]

im running into this quite often with evidence. is there a workaround form duckdb-wasm side?

[carlopi]

What is the value of the 'Content-Type' header for the parquet file being fetched ?

To get that, even just opening browser console, select network, instruct duckdb-wasm to fetch the relevant resource, select the relevant row clicking on it, then there is a Headers tab, look for 'Content-Type'.

If that is 'text/plain', it might be connected to this problem: #1580 that is connected to a problem in the spec + implementation of it by Web-browsers.

---

Independently of the first question question, couple of other ones.
Do you contol the server ?
Can you share a URL that demo this (ideally with a repro within shell.duckdb.org)?

[fboerman]

hi @carlopi content-type is application/octet-stream when I start evidence and it starts pulling in parquet. I do not think its related to that issue given that that bug is firefox specific. this one is the other way around, being solely chrome and edge specific. I dont think I ever managed to get it in firefox.

yes I control the server, you can go to reports.coreflowbased.eu and register a free account to test it out. thats my website which from time to time throws this error. if you get a database timeout simply refresh the site thats a different bug

[carlopi]

That bug is 2 bugs in one, one in Chrome and one in Firefox.

But that's not so relevant. I have no idea how to reproduce within the setting of your website, if you can give some instructions, I might be able to give it a try tomorrow.

[timothyhoward]

@carlopi Content-Type for the Parquet files is application/octet-stream here as well. I feel this is definitely a caching issue, as when the cache is disabled in Chrome/Edge, everything runs correctly.

Here's where the exception occurs. I tried looking up the status code 42 but couldn't find a reference.

[timothyhoward]

Can also confirm, the application works perfectly in Firefox on the same system (Windows 10), so it's confined to Edge and Chrome on Windows. Seems to be the exact same issue as the one @ericemc3 has described in the DuckDB Shell.

Edit: I've mitigated the issue on my application by disabling caching through the 'Cache-Control' header for parquet files. Error does not appear in Chrome/Edge with caching on Parquet files off.