Skip to content

Commit

Permalink
Rename followup (bluesky#482)
Browse files Browse the repository at this point in the history
* Follow up: s/node/container in structures explanation

* Rename create_node to create_container.

* Another manual pass through.
  • Loading branch information
danielballan authored Jul 5, 2023
1 parent 465c66c commit 815a7db
Show file tree
Hide file tree
Showing 21 changed files with 75 additions and 73 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ any HTTP client.
>>> client = from_uri("http://localhost:8000")

>>> client
<Node {'short_table', 'long_table', 'structured_data', ...} ~10 entries>
<Container {'short_table', 'long_table', 'structured_data', ...} ~10 entries>

>>> list(client)
'big_image',
Expand Down
27 changes: 14 additions & 13 deletions docs/source/explanations/structures.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Structures

Tiled *Adapters* provide data in one of a fixed group of standard *structure families*.
Tiled describes data in one of a fixed group of standard *structure families*.
These are *not* Python-specific structures. They can be encoded in standard,
language-agnostic formats and transferred from the service to a client in
potentially any language.
Expand All @@ -13,7 +13,7 @@ The structure families are:
* sparse --- a sparse array (i.e. an array which is mostly zeros)
* dataframe --- tabular data, as in [Apache Arrow](https://arrow.apache.org) or
[pandas](https://pandas.pydata.org/)
* node --- a grouping of other structures, akin to a dictionary or a directory
* container --- a of other structures, akin to a dictionary or a directory

Support for sparse arrays and [Awkward Array](https://awkward-array.org/) are
planned.
Expand Down Expand Up @@ -282,20 +282,20 @@ Both of the concepts (and their names) are borrowed directly from
dask.dataframe. They should enable any client, including in languages other than
Python, to perform the same function.

### Node
### Container

The node structure is a container for other structures. It may be compared to a
directory, a JSON object, a Python dictionary, or an HDF5 Group. Nodes may contain
other nodes, any other structure, or a mixture.
This structure is a container for other structures. It may be compared to a
directory, a JSON object, a Python dictionary, or an HDF5 Group. Containers may
contain other containers, any other structure, or a mixture.

Some nodes contain a small number of children, easy to list in a single request,
Some may contain a small number of nodes, easy to list in a single request,
while others may contain many listed via multiple paginated requests. Some Tiled
deployments currently in use contain nodes with with up to hundreds of thousands
of children.
deployments currently in use have containers with up to hundreds of thousands
of nodes.

Typically, a node's structures tell us only how many children it has (`count`).
Typically, a container's structure tell us only how many nodes it contains (`count`).
The `contents` key is typically set to `null`, which indicates that we will need
a separate request to fetch information of this node's children.
a separate request to fetch information about each child node.

```json
{
Expand All @@ -304,8 +304,9 @@ a separate request to fetch information of this node's children.
}
```

In certain cases, it is efficient to in-line all the information about the node's children
(their metadata, structure, and more) in a single response.
In certain cases, it is efficient to in-line all the information about the
container's contents (their metadata, structure, and more) in a single
response.


```json
Expand Down
4 changes: 2 additions & 2 deletions docs/source/how-to/client-logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@ Requests (`->`) and responses (`<-`) will now be logged to the console, like so.
16:49:22.572 <- 200 server:nginx/1.18.0 (Ubuntu) date:Tue, 01 Feb 2022 21:49:22 GMT content-type:application/x-msgpack content-length:292 connection:keep-alive etag:821dd2a8b431ecd016f94cacd44af74f server-timing:tok;dur=0.0, pack;dur=0.0, app;dur=3.8

>>> t = c['generated']['short_table']
16:58:22.589 -> GET 'https://tiled-demo.blueskyproject.io/node/search/?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=generated&sort=' 'host:tiled-demo.blueskyproject.io' 'accept:application/x-msgpack' 'accept-encoding:gzip,blosc' 'connection:keep-alive' 'user-agent:python-tiled/0.1.0a49.post0.dev0+g6dd1e5f' 'cookie:tiled_csrf=-fyaLez0YkradgcEVYBJh4QotR5MNyzouV0SV0NWHmM'
16:58:22.589 -> GET 'https://tiled-demo.blueskyproject.io/search/?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=generated&sort=' 'host:tiled-demo.blueskyproject.io' 'accept:application/x-msgpack' 'accept-encoding:gzip,blosc' 'connection:keep-alive' 'user-agent:python-tiled/0.1.0a49.post0.dev0+g6dd1e5f' 'cookie:tiled_csrf=-fyaLez0YkradgcEVYBJh4QotR5MNyzouV0SV0NWHmM'
16:58:22.635 <- 200 server:nginx/1.18.0 (Ubuntu) date:Tue, 01 Feb 2022 21:58:22 GMT content-type:application/x-msgpack content-length:502 connection:keep-alive etag:53a7b8a84ec504259a8c29903a25ade0 server-timing:tok;dur=0.0, pack;dur=0.0, app;dur=5.9
16:58:22.638 -> GET 'https://tiled-demo.blueskyproject.io/node/search/generated?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=short_table&sort=' 'host:tiled-demo.blueskyproject.io' 'accept:application/x-msgpack' 'accept-encoding:gzip,blosc' 'connection:keep-alive' 'user-agent:python-tiled/0.1.0a49.post0.dev0+g6dd1e5f' 'cookie:tiled_csrf=-fyaLez0YkradgcEVYBJh4QotR5MNyzouV0SV0NWHmM'
16:58:22.638 -> GET 'https://tiled-demo.blueskyproject.io/search/generated?filter%5Blookup%5D%5Bcondition%5D%5Bkey%5D=short_table&sort=' 'host:tiled-demo.blueskyproject.io' 'accept:application/x-msgpack' 'accept-encoding:gzip,blosc' 'connection:keep-alive' 'user-agent:python-tiled/0.1.0a49.post0.dev0+g6dd1e5f' 'cookie:tiled_csrf=-fyaLez0YkradgcEVYBJh4QotR5MNyzouV0SV0NWHmM'
16:58:22.681 <- 200 server:nginx/1.18.0 (Ubuntu) date:Tue, 01 Feb 2022 21:58:22 GMT content-type:application/x-msgpack content-length:944 connection:keep-alive etag:8d81b7891000606ceeb87fa89689c045 content-encoding:gzip vary:Accept-Encoding server-timing:acl;dur=0.0, tok;dur=0.1, pack;dur=0.0, compress;dur=0.1;ratio=4.5, app;dur=12.0

>>> t.read()
Expand Down
4 changes: 2 additions & 2 deletions docs/source/how-to/custom-clients.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ In the Python client, when a user accesses a given item, Tiled inspects the
item to decide what type of object to use to represent it.
In simple cases, this is just based on the `structure_family`: `"array"` goes
to `tiled.client.array.ArrayClient`; `"dataframe"` goes to
`tiled.client.dataframe.DataFrameClient`; `"node"` goes to
`tiled.clide.node.Container`. Those classes then manage further communication
`tiled.client.dataframe.DataFrameClient`; `"container"` goes to
`tiled.clide.container.Container`. Those classes then manage further communication
with Tiled server to access their contents.

Each item always has exactly one `structure_family`, and it's always from a
Expand Down
2 changes: 1 addition & 1 deletion docs/source/how-to/custom-export-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Tiled server can provide data---in whole or in part---in a variety of
formats. See {doc}`../tutorials/export` for a list of the formats supported out
of the box for each structure family (`"array"`, `"dataframe"`, `"node"`, ...).
of the box for each structure family (`"array"`, `"dataframe"`, `"container"`, ...).

This set of formats can easily be extended. A complete working example is
included in the tiled source tree at `example_configs/custom_export_formats`.
Expand Down
8 changes: 4 additions & 4 deletions docs/source/reference/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ The content of `tokens.json` looks like
Make an authenticated request using that access token.

```
$ http GET :8000/api/v1//metadata/ "Authorization:Bearer `jq -r .access_token tokens.json`"
$ http GET :8000/api/v1/metadata/ "Authorization:Bearer `jq -r .access_token tokens.json`"
HTTP/1.1 200 OK
content-length: 239
content-type: application/json
Expand All @@ -105,8 +105,8 @@ set-cookie: tiled_csrf=1-Cpa1WcwggakZ91FtNsscjM8VO1N1znmuILlL5hGY8; HttpOnly; Pa
},
"id": "",
"links": {
"search": "http://localhost:8000/api/v1/node/search/",
"self": "http://localhost:8000/api/v1//metadata/"
"search": "http://localhost:8000/api/v1/search/",
"self": "http://localhost:8000/api/v1/metadata/"
},
"meta": null,
"type": "tree"
Expand All @@ -121,7 +121,7 @@ When the access token expires (after 15 minutes, by default) requests will be
rejected like this.

```
$ http GET :8000/api/v1//metadata/ "Authorization:Bearer `jq -r .access_token tokens.json`"
$ http GET :8000/api/v1/metadata/ "Authorization:Bearer `jq -r .access_token tokens.json`"
HTTP/1.1 401 Unauthorized
content-length: 53
content-type: application/json
Expand Down
18 changes: 9 additions & 9 deletions docs/source/reference/http-api-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,27 +14,27 @@ and {doc}`authentication` for details.

## Overview

The routes are generally spelled like ``GET /{action}/{path}/``, like GitHub
The routes are generally spelled like ``GET /api/v1/{action}/{path}/``, like GitHub
repository URLs, with the path following the structure of the Tree
entries.

The ``GET //metadata/{path}`` route provides the metadata about one node.
The ``GET /node/search/{path}`` route provides paginated access to the children of
a given node, with optional filtering (search). The ``GET /node/full/{path}`` route
The ``GET /api/v1/metadata/{path}`` route provides the metadata about one node.
The ``GET /api/v1/search/{path}`` route provides paginated access to the children of
a given node, with optional filtering (search). The ``GET /api/v1/node/full/{path}`` route
provides all the metadata and data below a given node.

Specialized data access routes ``GET /array/block/{path}``, ``GET /array/full/{path}``,
and ``GET /dataframe/partition/{path}`` provide options for slicing and sub-selection
Specialized data access routes ``GET /api/v1/array/block/{path}``, ``GET /api/v1/array/full/{path}``,
and ``GET /api/v1/dataframe/partition/{path}`` provide options for slicing and sub-selection
specific to arrays and dataframes. Generic clients, like a web browser,
should use the "full" routes, which send the entire (sliced) result in one
response. More sophisticated clients with some knowledge of Tiled may use the
other routes, which enable parallel chunk-based access.

The root route, `GET /` provides general information about the server and the formats
The root route, `GET /api/v1/` provides general information about the server and the formats
and authentication providers it supports.

Depending on the server configuration, there may be authentication-related routes
nested under `/auth/`.
nested under `/api/v1/auth/`.

## Reference

Expand All @@ -46,7 +46,7 @@ or, to work fully locally, start the Tiled server with the demo
Tree from a Terminal

```
tiled serve pyobject --public tiled.examples.generated:tree
tiled serve demo
```

and navigate your browser to http://localhost:8000/docs.
14 changes: 7 additions & 7 deletions docs/source/reference/python-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ These are functions for constructing a client object.
tiled.client.from_profile
```

## Client Node
## Client Container

The Node interface extends the ``collections.abc.Mapping`` (i.e. read-only
The Container interface extends the ``collections.abc.Mapping`` (i.e. read-only
dict) interface, so it supports these standard "magic methods":

* `__getitem__` (lookup by key with `[]`)
Expand All @@ -37,9 +37,9 @@ The views returned by `.keys()`, `.items()`, and `.values()`
support efficient random access---e.g.

```py
node.values()[3]
node.values()[-1]
node.values()[:3]
c.values()[3]
c.values()[-1]
c.values()[:3]
```

and several convenience methods:
Expand All @@ -57,7 +57,7 @@ and several convenience methods:

Likewise for `.keys()` and `.items()`.

Beyond the Mapping interface, Node adds the following attributes
Beyond the Mapping interface, Container adds the following attributes

```{eval-rst}
.. autosummary::
Expand All @@ -69,7 +69,7 @@ Beyond the Mapping interface, Node adds the following attributes
tiled.client.container.Container.specs
```

It adds these methods, which return a new Node instance.
It adds these methods, which return a new Container instance.

```{eval-rst}
.. autosummary::
Expand Down
6 changes: 3 additions & 3 deletions docs/source/reference/scopes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ with restricted scopes.

* `read:metadata` --- List and search metadata.
* `read:data` --- Fetch (array, dataframe) data.
* `create` --- Create a new node. This is not yet used by Tiled itself. It is made available for use by experimental externally-developed adapters that support writing.
* `write:metadata` --- Write metadata. This is not yet used by Tiled itself. It is made available for use by experimental externally-developed adapters that support writing.
* `write:data` --- Write (array, dataframe) data. This is not yet used by Tiled itself. It is made available for use by experimental externally-developed adapters that support writing.
* `create` --- Create a new node.
* `write:metadata` --- Write metadata.
* `write:data` --- Write (array, dataframe) data.
* `apikeys` --- Manage API keys for the currently-authenticated user or service.
* `metrics` --- Access Prometheus metrics.
* `admin:apikeys` --- Manage API keys on behalf of any user or service.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ receive a response. For example...
>>> client = from_uri("http://localhost:8000", cache=Cache.in_memory(2e9))

>>> client
<Node {'big_image', 'small_image', 'tiny_image', 'tiny_cube', ...} ~11 entries>
<Container {'big_image', 'small_image', 'tiny_image', 'tiny_cube', ...} ~11 entries>

>>> client['long_table']
<DataFrameClient ['A', 'B', 'C']>
Expand Down
4 changes: 2 additions & 2 deletions docs/source/tutorials/login.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ and you can access data.

```
>>> c
<Node {'big_image', 'small_image', 'medium_image', ...} ~13 entries>
<Container {'big_image', 'small_image', 'medium_image', ...} ~13 entries>
```

Next, quit Python and start it fresh.
Expand All @@ -63,7 +63,7 @@ Next, quit Python and start it fresh.
>>> from tiled.client import from_uri
>>> c = from_uri("https://tiled-demo.blueskyproject.io")
>>> c
<Node {'big_image', 'small_image', 'medium_image', ...} ~13 entries>
<Container {'big_image', 'small_image', 'medium_image', ...} ~13 entries>
```

Notice that you are _not_ prompted to log in again. The login process
Expand Down
12 changes: 6 additions & 6 deletions docs/source/tutorials/navigation.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,16 +47,16 @@ however many fit on one line.

```python
>>> client
<Node {'big_image', 'small_image', 'tiny_image', 'tiny_cube', ...} ~11 entries>
<Container {'big_image', 'small_image', 'tiny_image', 'tiny_cube', ...} ~11 entries>
```

Nodes act like (nested) mappings in Python. All the (read-only) methods
that work on Python dictionaries work on Nodes. We can lookup a specific
Containers act like (nested) mappings in Python. All the (read-only) methods
that work on Python dictionaries work on Containers. We can lookup a specific
value by its key

```python
>>> client['structured_data']
<Node {'image_with_coords', 'xarray_dataset'}>
<Container {'image_with_coords', 'xarray_dataset'}>
```

list all the keys
Expand Down Expand Up @@ -93,7 +93,7 @@ for key, value in client.items():
...
```

Nodes also support efficient list-like access. This is useful for quickly
Containers also support efficient list-like access. This is useful for quickly
looking at a couple or efficiently grabbing batches of items, especially if you
need to start from the middle.

Expand Down Expand Up @@ -144,4 +144,4 @@ DictView({})
DictView({'animal': 'dog', 'color': 'red'})
```

See a later tutorial for how to search Nodes with queries.
See a later tutorial for how to search Containers with queries.
6 changes: 3 additions & 3 deletions docs/source/tutorials/search.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,22 +44,22 @@ anywhere in the metadata.
>>> from tiled.queries import FullText

>>> client.search(FullText("dog"))
<Node {'short_table', 'long_table'}>
<Container {'short_table', 'long_table'}>
```

The result has a subset of the contents of the original.
Searches may be chained to progressively narrow results:

```python
>>> client.search(FullText("dog")).search(FullText("red"))
<Node {'short_table'}>
<Container {'short_table'}>
```

If there no matches, the result is an empty Node:

```python
>>> client.search(FullText("something that will not be found"))
<Node {}>
<Container {}>
```

## More Queries
Expand Down
6 changes: 3 additions & 3 deletions docs/source/tutorials/serving-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,10 @@ disk, and we can slice and access the data.

```python
>>> client
<Node {'more', 'b', 'a', 'c', ...} ~7 entries>
<Container {'more', 'b', 'a', 'c', ...} ~7 entries>

>>> client['more']
<Node {'d'}>
<Container {'d'}>

>>> client['more']['d']
<ArrayClient>
Expand All @@ -74,7 +74,7 @@ array([[1., 1., 1., ..., 1., 1., 1.],
[1., 1., 1., ..., 1., 1., 1.]])

>>> client['tables']
<Node {'Sheet 1', 'Sheet 2'}>
<Container {'Sheet 1', 'Sheet 2'}>

>>> client['tables']['Sheet 1']
<DataFrameClient ['A', 'B']>
Expand Down
2 changes: 1 addition & 1 deletion scripts/json_indexes.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

async def test(a):
for i in range(100):
await a.create_node(
await a.create_container(
metadata={
"number": i,
"number_as_string": str(i),
Expand Down
Loading

0 comments on commit 815a7db

Please sign in to comment.