Skip to content

Google Earth Engine integration notebook #220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jan 20, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
324 changes: 324 additions & 0 deletions jupyter-notebooks/gee-integration/gee-integration.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,324 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Order and Delivery to Google Earth Engine\n",
"\n",
"In this notebook we're going to cover how to order data using the [Orders API](https://developers.planet.com/apis/orders/) and deliver it to [Google Earth Engine (GEE)](https://earthengine.google.com/) using [Planet’s GEE Delivery Integration](https://developers.planet.com/docs/integrations/gee/).\n",
"\n",
"This example demonstrates how to:\n",
"1. Define the data to be ordered\n",
"2. Build a cloud delivery configuration object (`delivery_config`), which tells the Orders API where to deliver the data\n",
"3. Build an order request to be sent off to the Orders API, `iowa_order`\n",
"4. Create the order and have it deliver to your GEE project\n",
"\n",
"**Prerequisites:** \n",
"- [Planet's Python SDK 2.0](https://github.com/planetlabs/planet-client-python) installed and initialized in your environment. Please follow instructions in [our docs](https://planet-sdk-for-python-v2.readthedocs.io/en/latest/get-started/quick-start-guide/) to authenticate your account with Planet servers. (**This notebook is not compatible with earlier versions of Planet's Python SDK**)\n",
"- An AOI - `iowa_aoi`\n",
"- Item ID(s) - `iowa_images`\n",
"- A GEE project with EE API enabled - `planet-devrel-dev`\n",
"- A pre-existing GEE [ImageCollection](https://developers.google.com/earth-engine/guides/ic_creating) - `gee-integration-testing`\n",
"- An account with a download quota. Not sure if you have download quota? Please [get in touch](https://www.planet.com/contact-sales/)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import planet"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define data to be ordered"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# The area of interest (AOI) defined as a polygon\n",
"iowa_aoi = {\n",
" \"type\":\n",
" \"Polygon\",\n",
" \"coordinates\": [[[-91.198465, 42.893071], [-91.121931, 42.893071],\n",
" [-91.121931, 42.946205], [-91.198465, 42.946205],\n",
" [-91.198465, 42.893071]]]\n",
"}\n",
"\n",
"# The item IDs we wish to order\n",
"iowa_images = ['20200925_161029_69_2223', '20200925_161027_48_2223']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define cloud delivery location\n",
"This is the *key concept* of this notebook. Here, we are defining a cloud delivery configuration object, `cloud_config`, where we are defining the destination to be Google Earth Engine, in the project named `planet-devrel-dev`, in the ImageCollection named `gee-integration-testing`."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# Google Earth Engine configuration\n",
"cloud_config = planet.order_request.google_earth_engine(\n",
" project='planet-devrel-dev', collection='gee-integration-testing')\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#nit might be nice to have the GEE delivery options as consts at the top of the notebook

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this were a regular script I would 100% do this, but I fear that would be a little inconsistent with how we typically lay out our notebooks. Thoughts on this @mkshah605?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be open to it! It might make sense so users can easily change their credentials. It's similar to how I laid out this Subscriptions notebook, where people can define their bucket and gcs key in cell #2. Definitely open to your thoughts!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've thought about it and we only use the project and collection names a single time, just in this cell, and it feels a bit unnecessary to define these as constants. @mkshah605, it makes sense in your subscriptions notebook, as you use bucket and gcs_key more than once in your notebook, but in this case I seem to justify creating two more variables. Are you opposed to leaving them as is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that makes sense. I am good with leaving them in here

"# Order delivery configuration\n",
"delivery_config = planet.order_request.delivery(cloud_config=cloud_config)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build a simple order request\n",
"Here we are building an order request, where we specify the products we wish to order. Spesifically, here we are requesting the IDs defined in `iowa_images`, as `analytic_udm2` assets, from the Planet Scope Scene product, `PSScene`."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'name': 'iowa_order', 'products': [{'item_ids': ['20200925_161029_69_2223', '20200925_161027_48_2223'], 'item_type': 'PSScene', 'product_bundle': 'analytic_sr_udm2'}], 'delivery': {'google_earth_engine': {'project': 'planet-devrel-dev', 'collection': 'gee-integration-testing'}}}\n"
]
}
],
"source": [
"# Product description for the order request\n",
"data_products = [\n",
" planet.order_request.product(item_ids=iowa_images,\n",
" product_bundle='analytic_sr_udm2',\n",
" item_type='PSScene')\n",
"]\n",
"\n",
"# Build the order request\n",
"iowa_order = planet.order_request.build_request(name='iowa_order',\n",
" products=data_products,\n",
" delivery=delivery_config)\n",
"\n",
"print(iowa_order)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create and deliver the order\n",
"Define a function that will create an order and update you with its progress. In this case, since we specified the delivery destination to be Google Earth Engine with `delivery_config`, it will also wait for it to be delivered to your project."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"async def create_and_deliver_order(order_request, client):\n",
" '''Create and deliver an order.\n",
"\n",
" Parameters:\n",
" order_request: An order request\n",
" client: An Order client object\n",
" '''\n",
" with planet.reporting.StateBar(state='creating') as reporter:\n",
" # Place an order to the Orders API\n",
" order = await client.create_order(order_request)\n",
" reporter.update(state='created', order_id=order['id'])\n",
" # Wait while the order is being completed\n",
" await client.wait(order['id'],\n",
" callback=reporter.update_state,\n",
" max_attempts=0)\n",
"\n",
" # Grab the details of the orders\n",
" order_details = await client.get_order(order_id=order['id'])\n",
"\n",
" return order_details"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create and deliver the order, `iowa_order`, to the GEE project, `planet-devrel-dev`, in the ImageCollection, `gee-integration-testing`."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"54:18 - order befd89fa-d314-4194-9001-1ac656fc8d27 - state: success\n"
]
}
],
"source": [
"async with planet.Session() as ps:\n",
" # The Orders API client\n",
" client = planet.OrdersClient(ps)\n",
" # Create the order and deliver it to GEE\n",
" order_details = await create_and_deliver_order(iowa_order, client)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Display the results for the first item data in the order."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'delivery': 'success', 'expires_at': '2022-12-01T00:37:11.420Z', 'location': 'https://api.planet.com/compute/ops/download/?token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Njk4NTUwMzEsInN1YiI6IllzV0ZqTDE4c04zeXYyZE1HQmNjeEVjeDFCQ0ZHeVgrUmlZWG9UYkU1Q3lFL2lMblE2VHhVMzBMc3JuczR6aEI5TUZvTmVTc0pHUFEzQ3lkc1hRMDVRPT0iLCJ0b2tlbl90eXBlIjoiZG93bmxvYWQtYXNzZXQtc3RhY2siLCJhb2kiOiIiLCJhc3NldHMiOlt7Iml0ZW1fdHlwZSI6IiIsImFzc2V0X3R5cGUiOiIiLCJpdGVtX2lkIjoiIn1dLCJ1cmwiOiJodHRwczovL3N0b3JhZ2UuZ29vZ2xlYXBpcy5jb20vY29tcHV0ZS1vcmRlcnMtbGl2ZS8yMDIwMDkyNV8xNjEwMjlfNjlfMjIyM18zQl9BbmFseXRpY01TX1NSLnRpZj9YLUdvb2ctQWxnb3JpdGhtPUdPT0c0LVJTQS1TSEEyNTZcdTAwMjZYLUdvb2ctQ3JlZGVudGlhbD1jb21wdXRlLWdjcy1zdmNhY2MlNDBwbGFuZXQtY29tcHV0ZS1wcm9kLmlhbS5nc2VydmljZWFjY291bnQuY29tJTJGMjAyMjExMzAlMkZhdXRvJTJGc3RvcmFnZSUyRmdvb2c0X3JlcXVlc3RcdTAwMjZYLUdvb2ctRGF0ZT0yMDIyMTEzMFQwMDM3MTFaXHUwMDI2WC1Hb29nLUV4cGlyZXM9ODYzOTlcdTAwMjZYLUdvb2ctU2lnbmF0dXJlPWE2OWQxNjE4ZGMwZDk0ODY3Y2I2N2EzY2U3MGUyNTU0M2VhODMyMmU2ZTFmZjFjMjE1NWVlZGU4OGQyYTgyMWVhMWU2MGEzNzRlYWQ5MWVhYjQwMzQ4NmQ4NzNjNzJhZjEzYWEwOTI4ZTVmMmFkNjhlYzU2M2UyZGRiZWQ3ZWZkZWFjOGUyNGJiZDM5MzRmOWFkMTNlMmM5ZjRiMmU0NmUxNjkyYzYzMWY4ZWM4Mjc1ZWRmYzg3ZjE0ZDkxMmNmYjUxMDE2N2IwM2FhMTY5MDRjY2YwNTY1ZjU4MDMzYTg3N2Y1Mzg3ZjI1YjU5ZGFjMTg4MDM1MWI3NGFlMmQyYTgxMWVhNjQ2ZDcwNGIyNTZkZmY5MGIxNWVmYTc2MTBlZWZlMjExMDI4ZTQ0ZDc1MjYxOGY4YzM1MzZkYWFkNmQ4MjJkNDVkZjNiMDNkZGUxOWQ2MDQwNjEzNzA0ZDQ3MWNkOGM5ZGQ1MTFlZWRjYTNjYzJjNWQyZDU4YWQ4YzcwZjhjNGZjOWFjYzBhNmY0NjkwY2I4ZTY0MTY0YTc3Yzc1NjZkMWY1NmI0NWEwOWU5YjY2N2QwYTJmYTM0NmFmYTYxMDQxN2NjODg5MWVlODkzNDhlMDlhNWFiNGQ3MmM0YTA2ZTZiNTk0NTMzMWVlYWQwYjJhNmUxMTFjYjJjMzYwXHUwMDI2WC1Hb29nLVNpZ25lZEhlYWRlcnM9aG9zdCIsInNvdXJjZSI6Ik9yZGVycyBTZXJ2aWNlIn0.FnaAXSRd_oeayLitsmws_dF6Uw-c_17fSSzrQfmAG1SzE7O_T4a1QKh8kAR8ijew0H5URUQm75qRLY6Swt4crQ', 'name': '20200925_161029_69_2223_3B_AnalyticMS_SR.tif'}\n"
]
}
],
"source": [
"print(order_details['_links']['results'][0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build an order request with clipping and harmonization applied\n",
"Here we are building an order request similar to the one above, but with [clipping](https://developers.planet.com/apis/orders/tools/#clip) and [harmonization](https://developers.planet.com/apis/orders/tools/#harmonization) applied. Namely, we are asking the Orders API to clip our images to the AOI, as defined by `iowa_aoi`, and radiometrically harmonizing the imagery with Sentinel-2's sensor. It's important to note that although the Orders API supports more tools, clipping and harmonization are the only two that are currently supported with GEE integration."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'name': 'iowa_order', 'products': [{'item_ids': ['20200925_161029_69_2223', '20200925_161027_48_2223'], 'item_type': 'PSScene', 'product_bundle': 'analytic_sr_udm2'}], 'delivery': {'google_earth_engine': {'project': 'planet-devrel-dev', 'collection': 'gee-integration-testing'}}, 'tools': [{'clip': {'aoi': {'type': 'Polygon', 'coordinates': [[[-91.198465, 42.893071], [-91.121931, 42.893071], [-91.121931, 42.946205], [-91.198465, 42.946205], [-91.198465, 42.893071]]]}}}, {'harmonize': {'target_sensor': 'Sentinel-2'}}]}\n"
]
}
],
"source": [
"# Clip images to the AOI's perimeter and harmonize the data with Dove Classic\n",
"tools = [\n",
" planet.order_request.clip_tool(iowa_aoi),\n",
" planet.order_request.harmonize_tool('Sentinel-2')\n",
"]\n",
"\n",
"# Build the order request\n",
"iowa_order = planet.order_request.build_request(name='iowa_order',\n",
" products=data_products,\n",
" delivery=delivery_config,\n",
" tools=tools)\n",
"\n",
"print(iowa_order)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create and deliver the clipped and harmonized data to `planet-devrel-dev` in `gee-integration-testing`."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"08:16 - order 37d82131-6a36-4b5c-a651-df30ca8ef945 - state: success\n"
]
}
],
"source": [
"async with planet.Session() as ps:\n",
" # The Orders API client\n",
" client = planet.OrdersClient(ps)\n",
" # Create the order and deliver it to GEE\n",
" order_details = await create_and_deliver_order(iowa_order, client)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Display the results for the first item data in the order."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'delivery': 'success', 'expires_at': '2022-12-01T01:05:06.668Z', 'location': 'https://api.planet.com/compute/ops/download/?token=eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2Njk4NTY3MDYsInN1YiI6IjdyZW9xVXRQaUQ5WVhaZkpwRU1wUkZQZVVBeFVzcDU2c3NaSnBTN0dMV3RZcWtZSFZJYWhwZXRISER6MHgxZTBTSWVYQk9CWjYrUXRDeDZFYnREVFlRPT0iLCJ0b2tlbl90eXBlIjoiZG93bmxvYWQtYXNzZXQtc3RhY2siLCJhb2kiOiIiLCJhc3NldHMiOlt7Iml0ZW1fdHlwZSI6IiIsImFzc2V0X3R5cGUiOiIiLCJpdGVtX2lkIjoiIn1dLCJ1cmwiOiJodHRwczovL3N0b3JhZ2UuZ29vZ2xlYXBpcy5jb20vY29tcHV0ZS1vcmRlcnMtbGl2ZS8yMDIwMDkyNV8xNjEwMjlfNjlfMjIyM18zQl91ZG0yX2NsaXAudGlmP1gtR29vZy1BbGdvcml0aG09R09PRzQtUlNBLVNIQTI1Nlx1MDAyNlgtR29vZy1DcmVkZW50aWFsPWNvbXB1dGUtZ2NzLXN2Y2FjYyU0MHBsYW5ldC1jb21wdXRlLXByb2QuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20lMkYyMDIyMTEzMCUyRmF1dG8lMkZzdG9yYWdlJTJGZ29vZzRfcmVxdWVzdFx1MDAyNlgtR29vZy1EYXRlPTIwMjIxMTMwVDAxMDUwNlpcdTAwMjZYLUdvb2ctRXhwaXJlcz04NjM5OVx1MDAyNlgtR29vZy1TaWduYXR1cmU9YTExNGE3Mjk3ZTNhZmEwYzRmNmU2ZDQxYjRkNzgxMWQ1YTkwNTcxNjI0MmMxMDUzZWI2OGYxZjE3NjhiY2E3NjNlNzg5ZmViZjAwMzYxYzI1NjgxN2FjMDU2MWY5NTQxMWU0OTI1MmEyMzBkZDQyNDUxNjcyYThlOWRhZDFmNDQwODE1NDVjZDlkYzYxNzI5MTFjZTAyYjkyMmE1OGM3OTJlYWI0NzdlMjY5MjU3ZTM1MWM3ZWVlM2Q0YWI5MjdkZjFjNjg0YWVmM2Q0OTc0NGEzODg1MWRhOTcxMTc2MTZlYjM3ZWM1NGZjMWE4MGIwZTU5MGJkNGUzZWFiMzEyYTY5MmIwZTg2ZmY0OWY0MzYxMjJmODBiZDFlYjIwYTgxZDdhYzMzYTY5YmY3OGExYmIwOWIzZGQ1YTE3MGYyYjE1YzJiZjM0MDk0NjJlY2IwYjIyNThhMjZkY2FkODlkMGExNDkyY2I5MGI1ZjViMmYwN2YyM2Q4MTY0YTljMDRhN2Q4ZWY0M2EzNTJjYTRlN2YwZDQ3MWYzNTZhNjQyMTU2Yzg1NTY3NDY0MDQ3MDUwZDc0NDgyMmU1MjgxZjkzZTM5ZWQ3NmU5YzdlYWZiZmVlY2QyMDgyNTE0MmQ0YmQ2OGRjMmZhNjc1NzNlZmU2ODc0ODk4M2I1YzZjM2MwNTNcdTAwMjZYLUdvb2ctU2lnbmVkSGVhZGVycz1ob3N0Iiwic291cmNlIjoiT3JkZXJzIFNlcnZpY2UifQ.cohH8ih049_CQF3NIEy-yAqAt6Vc_taEnXK5gn4Hwe6W3D3RqJAgoxyC7ZG3UJW0zmCdnq0653JKeAyScK94JQ', 'name': '20200925_161029_69_2223_3B_udm2_clip.tif'}\n"
]
}
],
"source": [
"print(order_details['_links']['results'][0])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.6 64-bit ('3.9.6')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "1bedda3fce59fa236ffac8164c02851c562f094c6d8f95a48784416ec3bbb813"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}