From 803eed4cd774458efc012f38699fe8daab55038d Mon Sep 17 00:00:00 2001 From: Zachary Deziel Date: Fri, 25 Aug 2023 15:52:31 -0700 Subject: [PATCH] Rename Hawaii demo and format Kahramanmaras demo Applied introduction and next steps structure to Kahramanmaras demo from the Hawaii one. Fixed typos in a few markdown cells. --- ...mo.ipynb => eoAPI_Maxar_Hawaii_demo.ipynb} | 0 .../eoAPI_Maxar_Kahramanmaras_demo.ipynb | 212 ++++++++++-------- 2 files changed, 118 insertions(+), 94 deletions(-) rename docs/src/notebooks/{eoAPI_Maxar_demo.ipynb => eoAPI_Maxar_Hawaii_demo.ipynb} (100%) diff --git a/docs/src/notebooks/eoAPI_Maxar_demo.ipynb b/docs/src/notebooks/eoAPI_Maxar_Hawaii_demo.ipynb similarity index 100% rename from docs/src/notebooks/eoAPI_Maxar_demo.ipynb rename to docs/src/notebooks/eoAPI_Maxar_Hawaii_demo.ipynb diff --git a/docs/src/notebooks/eoAPI_Maxar_Kahramanmaras_demo.ipynb b/docs/src/notebooks/eoAPI_Maxar_Kahramanmaras_demo.ipynb index c2866b1..46943ff 100644 --- a/docs/src/notebooks/eoAPI_Maxar_Kahramanmaras_demo.ipynb +++ b/docs/src/notebooks/eoAPI_Maxar_Kahramanmaras_demo.ipynb @@ -1,36 +1,43 @@ { "cells": [ { + "attachments": {}, "cell_type": "markdown", "id": "a0eb29a0", "metadata": {}, "source": [ - "## eoAPI Demo - Kahramanmaras earthquakes\n", + "## eoAPI Demo: Kahramanmaras earthquakes\n", "\n", "\n", - "This notebook will review the different [eoAPI](https://github.com/developmentseed/eoAPI) services using the latest Open data from Maxar acquired for the M7.8 and M7.5 Kahramanmaras earthquakes in Turkey on February 6 2023\n", + "### Background\n", "\n", - "More on the event: https://www.usgs.gov/news/featured-story/m78-and-m75-kahramanmaras-earthquake-sequence-near-nurdagi-turkey-turkiye\n", + "This notebook will review the different [eoAPI](https://github.com/developmentseed/eoAPI) services using the latest Open data from Maxar acquired for the M7.8 and M7.5 Kahramanmaras earthquakes in Turkey on February 6, 2023.\n", "\n", + "More on the event: https://www.usgs.gov/news/featured-story/m78-and-m75-kahramanmaras-earthquake-sequence-near-nurdagi-turkey-turkiye\n", "\n", - "### Maxar Open Data\n", + "### Objective\n", "\n", - "> Pre and post-event high-resolution satellite imagery in support of emergency planning, risk assessment, monitoring of staging areas and emergency response, damage assessment, and recovery. These images are generated using the Maxar ARD pipeline, tiled on an organized grid in analysis-ready cloud-optimized formats.\n", + "This notebook aims to demonstrate how [eoAPI](https://github.com/developmentseed/eoAPI) can be used to analyze Maxar's high-resolution satellite data to assess the Kahramanmaras earthquakes' impact.\n", "\n", + "### Maxar Open Data\n", "\n", + "Pre and post-event high-resolution satellite imagery in support of emergency planning, risk assessment, monitoring of staging areas and emergency response, damage assessment, and recovery. These images are generated using the Maxar ARD pipeline, tiled on an organized grid in analysis-ready cloud-optimized formats.\n", "\n", "### STAC and COGs\n", "\n", - "Maxar releases open data for select sudden-onset major crisis events. In addition to putting the data (as nice COGs) freely available on AWS, they are also adding STAC (static) metadata alongside the images. Having the STAC items already created makes ingestion into the PgSTAC database easy (we don't have to produce the items ourselves and thus have to read the images).\n", + "Maxar releases open data for select sudden-onset major crisis events. In addition to making the data (as nice COGs) freely available on AWS, they also add STAC (static) metadata alongside the images. Having the STAC items already created makes ingestion into the PgSTAC database easy (we don't have to produce the items ourselves and thus have to read the images).\n", "\n", - "To learn more about how we ingest Maxar OpenData STAC catalog into PgSTAC see https://github.com/vincentsarago/MAXAR_opendata_to_pgstac.\n", + "To learn more about ingesting the Maxar OpenData STAC catalog into PgSTAC see https://github.com/vincentsarago/MAXAR_opendata_to_pgstac.\n", "\n", + "---\n", "\n", - "### eoapi\n", + "> 📣 **Note: Important Information**\n", + "> - **Demo Endpoints**: This notebook uses the demo endpoints [stac.eoapi.dev](https://stac.eoapi.dev) and [raster.eoapi.dev](https://raster.eoapi.dev). These are not intended for production use.\n", + "> - **Authentication**: While the demo endpoints are currently open to the public, we may introduce authentication features in the future.\n", + "> - **Set Up Your Own Services**: If you're interested in setting up your own eoAPI services, you can easily do so. For a step-by-step guide, please check out the 'Getting Started' section of the [eoAPI GitHub repository](https://github.com/developmentseed/eoAPI).\n", + "> - **Breaking changes**: DevelopmentSeed does not guarantee that these eoAPI endpoints will stay `open`, and breaking changes might be introduced anytime.\n", "\n", - "For this notebook, we will use the DevelopmentSeed eoAPI's demo endpoints hosted at `stac.eoapi.dev` and `raster.eoapi.dev`. DevelopmentSeed do not guarantee that those endpoints will stay `open` and breaking changes might be introduced during time. \n", - "\n", - "You can check https://eoapi.dev if you want to learn how to deploy your own services." + "---" ] }, { @@ -71,13 +78,14 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "146cfd71", "metadata": {}, "source": [ "### Collection\n", "\n", - "if you look in `https://stac.eoapi.dev/collections` response you'll find one collection for the Kahramanmaras earthquake named `MAXAR_Kahramanmaras_turkey_earthquake_23`." + "If you look in `https://stac.eoapi.dev/collections` response, you'll find one collection for the Kahramanmaras earthquake named `MAXAR_Kahramanmaras_turkey_earthquake_23`." ] }, { @@ -101,11 +109,12 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "507f0afd", "metadata": {}, "source": [ - "Let's check the collection's metadata" + "Let's check the collection's metadata:" ] }, { @@ -117,7 +126,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "40217575311446f78031499bc14b6bca", + "model_id": "65fefc42d97146b9962488cf87af07de", "version_major": 2, "version_minor": 0 }, @@ -169,11 +178,12 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "5fd7a387", "metadata": {}, "source": [ - "Each collection can have spatial and temporal extents. As for the spatial extent, a collection can have multiple temporal extents but with its first one representing the combined min/max of all the intervals." + "Each collection can have spatial and temporal extents. As for the spatial extent, a collection can have multiple temporal extents, but its first one represents the combined min/max of all the intervals." ] }, { @@ -195,12 +205,12 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "0fa1db99", "metadata": {}, "source": [ "## Items\n", - "Let's check the items\n", "\n", "In this section, we will:\n", "\n", @@ -234,11 +244,12 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "ad0efb74", "metadata": {}, "source": [ - "As you can see below, the `/items` endpoints returned only 10 items. To return more data we need to either use the `paging` mechanism." + "As you can see below, the `/items` endpoints returned only 10 items. To return more data, we need to either use the `paging` mechanism." ] }, { @@ -274,16 +285,14 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 8, "id": "7f77306d", - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "8d1abdb3a1d34845ac95c310d5935de1", + "model_id": "2def97e8e551481082935449f9a6fa77", "version_major": 2, "version_minor": 0 }, @@ -291,7 +300,7 @@ "Map(center=[36.983902384345036, 36.89273358165191], controls=(ZoomControl(options=['position', 'zoom_in_text',…" ] }, - "execution_count": 12, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -320,24 +329,23 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "ec55f253", "metadata": {}, "source": [ "##### Item metadata \n", "\n", - "Each item should have an `id`, a `geometry`, some links to `Assets` and a set of properties.\n", + "Each item should have an `id`, a `geometry`, some links to `Assets`, and a set of properties.\n", "\n", "Item specification: https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md" ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 9, "id": "56ba6e0c", - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -670,7 +678,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 10, "id": "7f632955", "metadata": {}, "outputs": [ @@ -748,22 +756,21 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "844f4575", "metadata": {}, "source": [ "#### Find acquisition times\n", "\n", - "By definition, every item should have either a `datetime` or a `start/end_datetime` property. For the Maxar dataset, we are assuming that `datetime` is acquisition times." + "Every item should have either a `datetime` or a `start/end_datetime` property. For the Maxar dataset, we are assuming that `datetime` is acquisition times." ] }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 11, "id": "0d9013bb", - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -779,16 +786,17 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "a53e73ff", "metadata": {}, "source": [ - "Let's sort the items in two categories: `before` and `after` the event" + "Let's sort the items in two categories: `before` and `after` the event." ] }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 12, "id": "021248b8", "metadata": {}, "outputs": [ @@ -831,7 +839,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 13, "id": "29652a32", "metadata": {}, "outputs": [ @@ -876,7 +884,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 14, "id": "aca07238", "metadata": {}, "outputs": [ @@ -953,25 +961,23 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "7bc1a304", "metadata": {}, "source": [ "In eoAPI, we have a raster API connected to the PgSTAC database. The service is built using [titiler-pgstac](http://github.com/stac-utils/titiler-pgstac) and can be used to visualize `Item` or `Mosaics` (multiple items).\n", "\n", + "Endpoint: [https://raster.eoapi.dev](https://raster.eoapi.dev/)\n", "\n", - "Endpoint: https://raster.eoapi.dev\n", + "We know we have 4 assets for each item, and 3 are of `Cloud-Optimized` type. Let's use the raster API to visualize them.\n", "\n", - "\n", - "We know we have 4 assets for each item, and that 3 are of `Cloud-Optimized` type. Let's use the raster API to visualize them.\n", - "\n", - "\n", - "First, let's get the Raster metadata for each `raster` assets. The raster API will use the asset's `type` metadata to filter non-raster dataset (e.g. `application/geopackage+sqlite3`)." + "First, let's get the Raster metadata for each `raster` asset. The raster API will use the asset's `type` metadata to filter non-raster dataset (e.g. `application/geopackage+sqlite3`)." ] }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 15, "id": "e2a45bf1", "metadata": {}, "outputs": [ @@ -1025,8 +1031,8 @@ " \"green\",\n", " \"blue\"\n", " ],\n", - " \"height\": 17408,\n", " \"width\": 17408,\n", + " \"driver\": \"GTiff\",\n", " \"overviews\": [\n", " 2,\n", " 4,\n", @@ -1035,8 +1041,8 @@ " 32,\n", " 64\n", " ],\n", - " \"driver\": \"GTiff\",\n", - " \"count\": 3\n", + " \"count\": 3,\n", + " \"height\": 17408\n", "}\n", "visual 12 19\n", "ms_analytic 12 16\n", @@ -1059,14 +1065,14 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "3a87b045", "metadata": {}, "source": [ - "The `/collections/{collectionId}/items/{itemId}/info` endpoint returned metadata for 3 assets (the raster ones). We now know a bit more about each asset (datatype, zoom levels, number of bands), which can help us create tiles urls.\n", + "The `/collections/{collectionId}/items/{itemId}/info` endpoint returned metadata for 3 assets (the raster ones). We now know more about each asset (datatype, zoom levels, number of bands), which can help us create tiles urls.\n", "\n", - "\n", - "##### Asset on Map\n", + "### Asset on Map\n", "\n", "To visualize an asset on a Map, we need to construct a `Tile URL`. To ease the task we can use the raster's service `/collections/{collection_id}/items/{item_id}/tilejson.json` endpoint, but here are the requirements:\n", "\n", @@ -1078,11 +1084,9 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 16, "id": "b71ad07f", - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [ { "name": "stdout", @@ -1094,7 +1098,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "a8bd17c8ab0d48969c292e2c9a99d33d", + "model_id": "3272f8985af44e0189773f97e890e7b6", "version_major": 2, "version_minor": 0 }, @@ -1102,7 +1106,7 @@ "Map(center=[36.10519545142351, 36.404020998028585], controls=(ZoomControl(options=['position', 'zoom_in_text',…" ] }, - "execution_count": 23, + "execution_count": 16, "metadata": {}, "output_type": "execute_result" } @@ -1150,23 +1154,23 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "3a02b6ae", "metadata": {}, "source": [ "## Mosaics\n", "\n", - "As mentioned and shown with `titiler-pgstac`, we can visualize an item's asset, but the real power of the raster API is to be able to create a virtual mosaic dynamically and merge multiple items on the fly.\n", + "As mentioned and shown with `titiler-pgstac`, we can visualize an item's asset. Still, the raster API's real power is to create a virtual mosaic dynamically and merge multiple items on the fly.\n", "\n", "Learn more: http://github.com/stac-utils/titiler-pgstac.\n", "\n", - "\n", - "Let's create `mosaics` for `pre` and `post` event" + "Let's create `mosaics` for `pre` and `post` events." ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 17, "id": "57c276ec", "metadata": {}, "outputs": [ @@ -1294,14 +1298,14 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "6dd5f539", "metadata": {}, "source": [ "Explanation:\n", "\n", - "The `mosaic` correspond of a STAC Search query. Because we use `PgSTAC` backend we can make use of the `filter` extension to construct complex query: https://github.com/stac-api-extensions/filter\n", - "\n", + "The `mosaic` corresponds to an STAC Search query. Because we use `PgSTAC` backend, we can make use of the `filter` extension to construct complex queries: https://github.com/stac-api-extensions/filter\n", "\n", "```json\n", "{\n", @@ -1309,16 +1313,16 @@ " \"filter-lang\": 'cql2-json',\n", " // We tell PgSTAC to `register` a `search` request with the following filter:\n", " \"filter\": {\n", - " \"op\": 'and', \n", + " \"op\": 'and',\n", " \"args\": [\n", " // Item's collection HAS TO be in `[collection_id]`\n", " {\n", - " \"op\": \"in\", \n", + " \"op\": \"in\",\n", " \"args\": [{\"property\": \"collection\"}, [collection_id]]\n", " },\n", " // Filter Items that have datetime `lt` than the event date\n", " {\n", - " \"op\": \"lt\", \n", + " \"op\": \"lt\",\n", " \"args\": [\n", " {\"property\": \"datetime\"}, event_date\n", " ]\n", @@ -1331,35 +1335,36 @@ " },\n", " ],\n", " // titiler-pgstac accept some additional metadata\n", - " // https://stac-utils.github.io/titiler-pgstac/advanced/metadata/\n", - " // One is useful: `bounds`. When creating a mosaic, the tiler will have no idea \n", - " // where the items will be before trying to create each tile. To avoid trying to request \n", - " // tiles where we know we don't have any items, we can add the collection's extent to the \n", - " // mosaic metadata. \n", - " // The tiler service will then return the bounds in the tilejson document for the \n", + " // \n", + " // One is useful: `bounds`. When creating a mosaic, the tiler will have no idea\n", + " // where the items will be before trying to create each tile. To avoid trying to request\n", + " // tiles where we know we don't have any items, we can add the collection's extent to the\n", + " // mosaic metadata.\n", + " // The tiler service will then return the bounds in the tilejson document for the\n", " // client application.\n", " \"metadata\":{\n", " \"name\": \"Maxar Kahramanmaras - Pre event\",\n", " \"bounds\": bounds,\n", - " } \n", + " }\n", " ],\n", " },\n", "\n", "}\n", + "\n", "```\n", "\n", "API Response:\n", "\n", - "The raster service will return a `searchid` hash (also called `mosaic id`), which we can use to construct a tile URL.\n", + "The raster service will return a `searchid` hash (`mosaic id`), which we can use to construct a tile URL.\n", "\n", - "To create a valid tile URL we will again need to pass an `assets` parameter to tell the tiler which assets we want to visualize. We can also set the min/max zooms limits to avoid underzooming (openning to many files) and overzooming.\n", + "To create a valid tile URL, we will again need to pass an `assets` parameter to tell the tiler which assets we want to visualize. We can also set the min/max zoom limits to avoid underzooming (opening too many files) and overzooming.\n", "\n", - "See full list of options: https://stac-utils.github.io/titiler-pgstac/mosaic_endpoints/#tiles" + "See the complete list of options: https://stac-utils.github.io/titiler-pgstac/mosaic_endpoints/#tiles" ] }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 18, "id": "1638c679", "metadata": {}, "outputs": [ @@ -1373,7 +1378,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "ddd650b929f6435097dd63044dab0398", + "model_id": "924a36c6483e4f04b808d688cbf9d6bd", "version_major": 2, "version_minor": 0 }, @@ -1381,7 +1386,7 @@ "Map(center=[37.58145520587342, 36.9346202083822], controls=(ZoomControl(options=['position', 'zoom_in_text', '…" ] }, - "execution_count": 28, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } @@ -1431,7 +1436,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 19, "id": "c74df2f5", "metadata": {}, "outputs": [ @@ -1445,7 +1450,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "9f735f351f2f49dab084ad9fd2c4f5d5", + "model_id": "146a0b0664f949c593b5a5d58d48c0f8", "version_major": 2, "version_minor": 0 }, @@ -1453,7 +1458,7 @@ "Map(center=[37.58145520587342, 36.9346202083822], controls=(ZoomControl(options=['position', 'zoom_in_text', '…" ] }, - "execution_count": 29, + "execution_count": 19, "metadata": {}, "output_type": "execute_result" } @@ -1503,16 +1508,14 @@ }, { "cell_type": "code", - "execution_count": 38, + "execution_count": 20, "id": "b8d900b4", - "metadata": { - "scrolled": false - }, + "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "aef5e4a4499e4ab5ba680cd16aea4721", + "model_id": "9deeb7a0c43a44868cbc24271ee01dda", "version_major": 2, "version_minor": 0 }, @@ -1520,7 +1523,7 @@ "Map(center=[37.571788, 36.9191], controls=(ZoomControl(options=['position', 'zoom_in_text', 'zoom_in_title', '…" ] }, - "execution_count": 38, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } @@ -1560,12 +1563,33 @@ ] }, { - "cell_type": "code", - "execution_count": null, - "id": "bdac0711", + "attachments": {}, + "cell_type": "markdown", + "id": "8b9bc02d", "metadata": {}, - "outputs": [], - "source": [] + "source": [ + "## What's Next?\n", + "\n", + "### Spin Up Your Own eoAPI Instance\n", + "\n", + "You've seen what eoAPI can do with Maxar data in the context of the Turkey Earthquakes. Interested in setting up your own eoAPI service? It's straightforward! Follow the 'Getting Started' section of the [eoAPI GitHub repository](https://github.com/developmentseed/eoAPI) to get your instance up and running. This will give you greater control and customization options.\n", + "\n", + "### Contribute Your Data\n", + "\n", + "Consider contributing if you've used Maxar data for similar analyses or have other datasets that could benefit the community. Uploading your data to your eoAPI instance can provide more diverse examples and help in various applications.\n", + "\n", + "### Community-Driven Examples\n", + "\n", + "The examples you see here, including this notebook, are often community-driven. We encourage you to contribute your analyses, workflows, or visualizations. Your insights could be invaluable for helping others.\n", + "\n", + "### Have Questions? Join the Discussion!\n", + "\n", + "If you have questions, feedback, or want to engage with the eoAPI community, please join the [GitHub discussions](https://github.com/developmentseed/eoAPI/discussions). It's a great place to ask questions, share your experiences, and connect with others interested in eoAPI and its components. \n", + "\n", + "---\n", + "\n", + "Thank you for taking the time to go through this notebook." + ] } ], "metadata": { @@ -1584,7 +1608,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.17" + "version": "3.10.5" } }, "nbformat": 4,