Tutoral updates and small fixes (bluesky#748)

* docs: Small updates to tutorials to reflect current release.

* Updated `BaseClient.formats` to use the `dict` structure for specs.

Not triggered for ArrayClient, so added extra line to test.

* anyio.Event.set() is not an async function anymore

CHANGELOG.md

@@ -3,6 +3,13 @@ Write the date in place of the "Unreleased" in the case a new version is release
 
 # Changelog
 
+## Unreleased
+
+### Fixed
+
+- Updated `BaseClient.formats` to use the `dict` structure for specs.
+- The `tiled serve directory --watch` function was not compatible with recent `anyio`
+
 ## v0.1.0a122 (23 May 2024)
 
 ### Fixed

docs/source/tutorials/export.md

@@ -6,7 +6,7 @@ common formats for use by some external software (i.e. not Python).
 To follow along, start the Tiled server with example data from a Terminal.
 
 ```
-tiled serve pyobject --public tiled.examples.generated:tree
+tiled serve demo
 ```
 
 Now, in a Python interpreter, connect, with the Python client.
@@ -22,7 +22,7 @@ These are just a couple of the supported formats:
 
 ```python
 # Table
-client["short_talbe"].export("table.xlsx")  # Excel
+client["short_table"].export("table.xlsx")  # Excel
 client["short_table"].export("table.csv")  # CSV
 
 # Array
@@ -39,6 +39,7 @@ client["short_table"].export("table.csv", columns=["A", "B"])
 
 # Export an N-dimensional slice...
 client["medium_image"].export("numbers.csv", slice=[0])  # like arr[0]
+import numpy
 client["medium_image"].export("numbers.csv", slice=numpy.s_[:10, 100:200])  # like arr[:10, 100:200]
 ```
 

docs/source/tutorials/navigation.md

@@ -6,7 +6,7 @@ client.
 To follow along, start the Tiled server with example data from a Terminal.
 
 ```
-tiled serve pyobject --public tiled.examples.generated:tree
+tiled serve demo
 ```
 
 Now, in a Python interpreter, connect, with the Python client.
@@ -27,18 +27,18 @@ Tiled provides a utility for visualizing a nested structure.
 >>> tree(client)
 ├── big_image
 ├── small_image
+├── medium_image
+├── sparse_image
+├── awkward_array
 ├── tiny_image
 ├── tiny_cube
 ├── tiny_hypercube
-├── low_entropy
-├── high_entropy
 ├── short_table
 ├── long_table
-├── labeled_data
-│   └── image_with_dims
-└── structured_data
-    ├── image_with_coords
-    └── xarray_dataset
+├── wide_table
+├── structured_data
+│   ├── pets
+│   └── xarray_dataset
 ```
 
 Each (sub)tree displays the names of a couple of its entries---up to
@@ -47,7 +47,7 @@ however many fit on one line.
 
 ```python
 >>> client
-<Container {'big_image', 'small_image', 'tiny_image', 'tiny_cube', ...} ~11 entries>
+<Container {'big_image', 'small_image', 'medium_image', ...} ~16 entries>
 ```
 
 Containers act like (nested) mappings in Python. All the (read-only) methods
@@ -56,7 +56,7 @@ value by its key
 
 ```python
 >>> client['structured_data']
-<Container {'image_with_coords', 'xarray_dataset'}>
+<Container {'pets', 'xarray_dataset'}>
 ```
 
 list all the keys
@@ -65,15 +65,20 @@ list all the keys
 >>> list(client)
 ['big_image',
  'small_image',
+ 'medium_image',
+ 'sparse_image',
+ 'awkward_array',
  'tiny_image',
  'tiny_cube',
  'tiny_hypercube',
- 'low_entropy',
- 'high_entropy',
  'short_table',
  'long_table',
- 'labeled_data',
- 'structured_data']
+ 'wide_table',
+ 'structured_data',
+ 'flat_array',
+ 'low_entropy',
+ 'high_entropy',
+ 'dynamic']
 ```
 
 and loop over keys, values, or ``(key, value)`` pairs.
@@ -104,32 +109,32 @@ need to start from the middle.
 >>> client.keys().head()  # Access the first several keys.
 ['big_image',
  'small_image',
- 'tiny_image',
- 'tiny_cube',
- 'tiny_hypercube']
+ 'medium_image',
+ 'sparse_image',
+ 'awkward_array']
 
 >>> client.keys().head(3)  # Access the first N keys.
 ['big_image',
  'small_image',
- 'tiny_image']
+ 'medium_image']
 
 >>> client.keys()[1:3]  # Access just the keys for entries 1:3.
-['small_image', 'tiny_image']
+['small_image', 'medium_image']
 ```
 
 All the same methods work for values
 
 ```python
 >>> client.values()[1:3]  # Access the values (which may be more expensive).
-[<ArrayClient>, <ArrayClient>]
+[<ArrayClient shape=(300, 300) chunks=((300,), (300,)) dtype=float64>, <ArrayClient shape=(1000, 1000) chunks=((1000,), (1000,)) dtype=float64>]
 ```
 
 and `(key, value)` pairs ("items").
 
 ```python
 >>> client.items()[1:3]  # Access (key, value) pairs.
-[('small_image', <ArrayClient>),
-[('tiny_image', <ArrayClient>),
+[('small_image', <ArrayClient shape=(300, 300) chunks=((300,), (300,)) dtype=float64>),
+('medium_image', <ArrayClient shape=(1000, 1000) chunks=((1000,), (1000,)) dtype=float64>)]
 ```
 
 Each item has ``metadata``, which is a simple dict.

docs/source/tutorials/search.md

@@ -5,7 +5,7 @@ In this tutorial we will find a dataset by performing a search over metadata.
 To follow along, start the Tiled server with example data from a Terminal.
 
 ```
-tiled serve pyobject --public tiled.examples.generated:tree
+tiled serve demo
 ```
 
 ## Search Using the Python Client
@@ -44,15 +44,15 @@ anywhere in the metadata.
 >>> from tiled.queries import FullText
 
 >>> client.search(FullText("dog"))
-<Container {'short_table', 'long_table'}>
+<Container {'short_table', 'long_table', 'wide_table'}>
 ```
 
 The result has a subset of the contents of the original.
 Searches may be chained to progressively narrow results:
 
 ```python
 >>> client.search(FullText("dog")).search(FullText("red"))
-<Container {'short_table'}>
+<Container {'short_table', 'wide_table'}>
 ```
 
 If there no matches, the result is an empty Node:
@@ -64,5 +64,5 @@ If there no matches, the result is an empty Node:
 
 ## More Queries
 
-Above, use the `FullText` query. Tiled supports many queries;
+Above examples use the `FullText` query. Tiled supports many queries;
 see {doc}`../reference/queries`.

docs/source/tutorials/serving-files.md

@@ -15,7 +15,7 @@ This created a directory named ``example_files`` with some files and subdirector
 
 ```
 $ ls example_files
-another_table.csv  a.tif  b.tif  c.tif  even_more	more  tables.xlsx
+another_table.csv  a.tif  b.tif  c.tif  more  tables.xlsx
 ```
 
 The full structure looks like
@@ -26,7 +26,11 @@ The full structure looks like
 ├── b.tif
 ├── c.tif
 ├── more
-│   └── d.tif
+│   ├── A0001.tif
+│   ├── A0002.tif
+│   ├── A0003.tif
+│   ├── B0001.tif
+│   ├── B0002.tif
 │   └── even_more
 │       ├── e.tif
 │       └── f.tif
@@ -40,8 +44,8 @@ tiled serve directory --public example_files
 ```
 
 Tiled walks the directory, identifies files that it recognizes and has
-Readers for. It watches the directory for additions, removals, and changes to
-the file.
+Readers for. It can watch the directory for additions, removals, and changes to
+the file with option `--watch`.
 
 In a Python interpreter, connect with the Python client.
 
@@ -56,15 +60,15 @@ disk, and we can slice and access the data.
 
 ```python
 >>> client
-<Container {'more', 'b', 'a', 'c', ...} ~7 entries>
+<Container {'another_table', 'tables', 'c', 'a', 'b', 'more'}>
 
 >>> client['more']
-<Container {'d'}>
+<Container {'A', 'B', 'even_more'}>
 
->>> client['more']['d']
-<ArrayClient>
+>>> client['more']['A']
+<ArrayClient shape=(3, 100, 100) chunks=((1, 1, 1), (100,), (100,)) dtype=float64>
 
->>> client['more']['d'].read()
+>>> client['more']['A'][0]
 array([[1., 1., 1., ..., 1., 1., 1.],
        [1., 1., 1., ..., 1., 1., 1.],
        [1., 1., 1., ..., 1., 1., 1.],
@@ -77,7 +81,7 @@ array([[1., 1., 1., ..., 1., 1., 1.],
 <Container {'Sheet 1', 'Sheet 2'}>
 
 >>> client['tables']['Sheet 1']
-<DataFrameClient ['A', 'B']>
+<DataFrameClient>
 
 >>> client['tables']['Sheet 1'].read()
    A  B

docs/source/tutorials/slicing.md

@@ -6,7 +6,7 @@ need. We'll also use dask to delay download computation.
 To follow along, start the Tiled server with example data from a Terminal.
 
 ```
-tiled serve pyobject --public tiled.examples.generated:tree
+tiled serve demo
 ```
 
 Now, in a Python interpreter, connect with the Python client.
@@ -23,7 +23,7 @@ Navigate to an array dataset in the demo tree.
 
 ```python
 >>> client['medium_image']
-<ArrayClient>
+<ArrayClient shape=(1000, 1000) chunks=((1000,), (1000,)) dtype=float64>
 ```
 
 Slice ``[:]`` to read it. (This syntax may be familiar to h5py users.)
@@ -63,11 +63,10 @@ Navigate to a tabular dataset in the demo client.
 
 ```python
 >>> client['short_table']
-<DataFrameClient ['A', 'B', 'C']>
+<DataFrameClient>
 ```
 
-The columns are display in the output. You can also access them
-programmatically by listing them.
+You can access the columns by listing them.
 
 ```python
 list(client['short_table'])
@@ -95,10 +94,11 @@ index
 [100 rows x 3 columns]
 ```
 
-You may select a column or a list of columns.
+You may select a column or a list of columns, and access the column data array directly.
 
 ```python
 >>> client['short_table'].read(['A'])
+              A
 index
 0     0.100145
 1     0.634538
@@ -111,7 +111,7 @@ index
 97    0.620561
 98    0.909704
 99    0.456574
-Name: A, Length: 100, dtype: float64
+[100 rows x 1 columns]
 
 >>> client['short_table'].read(['A', 'B'])
               A         B
@@ -129,6 +129,9 @@ index
 99     0.456574  0.918859
 
 [100 rows x 2 columns]
+
+>>> client['short_table']['A']
+<ArrayClient shape=(100,) chunks=((100,),) dtype=float64>
 ```
 
 ## Dask

tiled/_tests/test_export.py

@@ -136,3 +136,4 @@ def test_path_as_Path_or_string(client, tmpdir):
 def test_formats(client):
     client.formats
     client["A"].formats
+    client["C"].formats

tiled/client/base.py

@@ -364,7 +364,7 @@ def formats(self):
         "List formats that the server can export this data as."
         formats = set()
         for spec in self.item["attributes"]["specs"]:
-            formats.update(self.context.server_info["formats"].get(spec, []))
+            formats.update(self.context.server_info["formats"].get(spec["name"], []))
         formats.update(
             self.context.server_info["formats"][
                 self.item["attributes"]["structure_family"]

tiled/client/register.py

@@ -490,7 +490,7 @@ async def watch(
         )
         # Signal that initial walk is complete.
         # Process any changes that were accumulated during the initial walk.
-        await initial_walk_complete_event.set()
+        initial_walk_complete_event.set()
 
 
 async def _watch(
@@ -506,7 +506,7 @@ async def _watch(
     def watch_filter(change, path):
         return settings.filter(Path(path))
 
-    await ready_event.set()
+    ready_event.set()
     backlog = []
     async for batch in watchfiles.awatch(
         path,