500 errors in Datasette 1.0 alpha

[heaversm]

When accessing https://latest-with-plugins.datasette.io/github/commits.Notebook I get the error Error 500 'NoneType' object has no attribute 'args'

[simonw]

This is because that instance is now running Datasette 1.0 alpha which broke compatibility with this plugin.

I'll see if I can get the plugin working for both versions.

[simonw]

Here's the problem:

datasette-export-notebook/datasette_export_notebook/__init__.py

Line 7 in ed06393

async def render_notebook(datasette, request, data, rows):

datasette-export-notebook/datasette_export_notebook/__init__.py

Lines 20 to 23 in ed06393

	total_count = None
	count = len(rows)
	if "filtered_table_rows_count" in data:
	total_count = data["filtered_table_rows_count"]

The plugin needs to know the total count of rows in the table. It used to be able to get that from filtered_table_rows_count - but that's not available by default now in Datasette 1.0. Instead we have this:

(Pdb) data
{'ok': True, 'next': None, 'rows': [{'rowid': 1, 'id': 0}, {'rowid': 2, 'id': 1}, ...

Adding ?_extra=count to the URL would have added a count field we can use: https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_extra=count

But... the render plugin hook doesn't have a mechanism to specify extras yet.

[simonw]

Here's how that total_count is actually used:

datasette-export-notebook/datasette_export_notebook/templates/export_notebook.html

Lines 43 to 64 in ed06393

	{% if allow_csv_stream and csv_stream_url and total_count and total_count > count %}
	<p>You can export all {{ "{:,}".format(total_count) }} rows using a single streaming CSV export like this:</p>
	<pre class="copyable">{{ pandas_stream_code }}</pre>
	{% endif %}
	<h2>Observable</h2>
	{% if cors_enabled %}
	<p>Import the data into a variable called <code>rows</code> like this:</p>
	<pre class="copyable">rows = d3.json(
	{{ json.dumps(json_url) }}
	)</pre>
	{% if allow_csv_stream and csv_stream_url and total_count and total_count > count %}
	<p>You can export all {{ "{:,}".format(total_count) }} rows using a single streaming CSV export like this:</p>
	<pre class="copyable">rows = d3.csv(
	{{ json.dumps(csv_stream_url) }},
	d3.autoType
	)</pre>
	{% endif %}
	{% else %}
	<p class="message-warning">Export to Observable is only available if Datasette is running with the <code>--cors</code> option.</p>
	{% endif %}

It's so the export page can say "You can export all 34,536 rows...".

I'm going to resolve this by teaching the plugin that if filtered_table_rows_count is not available it should just say "You can export all of the rows..."

And I'll use the presence of that next key to decide whether or not to show the streaming options.

Then I'll have CI run this both on Datasette 0.x and Datasette 1.x.

[simonw]

Huh, that did not fix the bug!

I got it to work on my local laptop using the 1.0 alphas and got the tests to pass against them, but...

https://latest-with-plugins.datasette.io/github/commits.Notebook is now running the latest version (see https://latest-with-plugins.datasette.io/-/plugins) is still returning the same error: