reviewed and edited introduction guide

Author: Atma Mani

…-output-with-the-spatial-dataframe.ipynb …roduction-to-the-spatial-dataframe.ipynbguide/05-working-with-the-spatial-dataframe/input-and-output-with-the-spatial-dataframe.ipynb renamed to guide/05-working-with-the-spatial-dataframe/introduction-to-the-spatial-dataframe.ipynb guide/05-working-with-the-spatial-dataframe/input-and-output-with-the-spatial-dataframe.ipynb renamed to guide/05-working-with-the-spatial-dataframe/introduction-to-the-spatial-dataframe.ipynb

@@ -4,20 +4,20 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Working with Data\n",
+    "# Introduction to the Spatial DataFrame\n",
     "\n",
-    "The [Spatial DataFrame](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#spatialdataframe) (SDF) creates a simple, intutive object that can easily manipulate geometric and attribute data without forcing you to use a full-fledged copy of any source information. The SDF is based on data structures inherently suited to data analysis, with natural operations for the filtering and inspecting of subsets of values which are fundamental to statistical manipulations. \n",
+    "The [Spatial DataFrame](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#spatialdataframe) (SDF) creates a simple, intutive object that can easily manipulate geometric and attribute data. The Spatial DataFarme extends the popular Pandas](https://pandas.pydata.org/) [DataFrames](http://pandas.pydata.org/pandas-docs/stable/dsintro.html#dataframe) with spatial abilities, allowing you to use intutive pandorable operations, on both the attribute as well as spatial columns. Thus the SDF is based on data structures inherently suited to data analysis, with natural operations for the filtering and inspecting of subsets of values which are fundamental to statistical and geographic manipulations.\n",
     "\n",
     "Data can be read from many **sources**, including shapefiles, [Pandas](https://pandas.pydata.org/) [DataFrames](http://pandas.pydata.org/pandas-docs/stable/dsintro.html#dataframe), feature classes, GeoJSON, and Feature Layers.\n",
     "\n",
-    "This document outlines some fundamentals of using the Spatial DataFrame object for working with GIS data.\n",
+    "This document outlines some fundamentals of using the `SpatialDataFrame` object for working with GIS data.\n",
     "\n",
-    "* [Accessing Published Data](#Accessing-Published-Data)\n",
-    " * [Reading Service Data](#Reading-Service-Data)\n",
+    "* [Accessing GIS Data](#Accessing-GIS-data)\n",
+    " * [Reading Web Layers](#Reading-Web-Layers)\n",
     " * [Reading Feature Layer Data](#Reading-Feature-Layer-Data)\n",
     "   * [Examining Feature Layer Content](#Example:-Examining-Feature-Layer-content)\n",
     "   * [Example: Feature Layer Query Results to a Spatial DataFrame](#Example:-Feature-Layer-Query-Results-to-a-Spatial-DataFrame)\n",
-    "* [Accessing Feature Class Data](#Accessing-Feature-Class-Data)\n",
+    "* [Accessing local GIS Data](#Accessing-local-GIS-data)\n",
     " * [Example: Reading a Shapefile](#Example:-Reading-a-Shapefile)\n",
     "* [Saving Spatial DataFrames](#Saving-Spatial-DataFrames)\n",
     " * [Export Options](#Export-Options)\n",
@@ -39,19 +39,18 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Accessing Published Data\n",
+    "## Accessing GIS data\n",
+    "GIS users need to work with both layers published on remote servers (web layers) and local data, but the ability to manipulate these datasets without permanentently copying the data is lacking.  The `SpatialDataFrame` solves this problem because it is an in-memory object that can read, write and manipulate geospatial data.\n",
     "\n",
-    "Software users need to work with both published data on remote servers and local data, but the ability to manipulate these datasets without permanentently copying the data is lacking.  The `Spatial DataFrame` solves this problem because it is an in-memory object that can read, write and manipulate geospatial data. \n",
-    "\n",
-    "The SDF integrates with Esri's [`ArcPy site-package`](http://pro.arcgis.com/en/pro-app/arcpy/get-started/what-is-arcpy-.htm) as well as the open source [`pyshp`](https://github.com/GeospatialPython/pyshp/), [`shapely`](https://github.com/Toblerity/Shapely) and [`fiona`](https://github.com/Toblerity/Fiona) packages. This means the ArcGIS API for Python SDF can use either of these geometry engines to provide you options for easily working with geospatial data regardless of your platform.  The SDF transforms data into the formats you desire so you can use Python functionality to analyze and visualize geographic information.  \n",
+    "The SDF integrates with Esri's [`ArcPy site-package`](http://pro.arcgis.com/en/pro-app/arcpy/get-started/what-is-arcpy-.htm) as well as the open source [`pyshp`](https://github.com/GeospatialPython/pyshp/), [`shapely`](https://github.com/Toblerity/Shapely) and [`fiona`](https://github.com/Toblerity/Fiona) packages. This means the ArcGIS API for Python SDF can use either of these geometry engines to provide you options for easily working with geospatial data regardless of your platform.  The SDF transforms data into the formats you desire so you can use Python functionality to analyze and visualize geographic information.\n",
     "\n",
     "Data can be read and scripted to automate workflows and just as easily visualized on maps in [`Jupyter notebooks`](../using-the-jupyter-notebook-environment/). The SDF can export data as feature classes or publish them directly to servers for sharing according to your needs.\n",
     "\n",
     "Let's explore some of the different options available with the versatile `SpatialDataFrame` object:\n",
     "\n",
-    "### Reading Service Data\n",
+    "### Reading Web Layers\n",
     "\n",
-    "[`Feature layers`](https://doc.arcgis.com/en/arcgis-online/share-maps/hosted-web-layers.htm) hosted on [**ArcGIS Online**](https://www.arcgis.com) provide a wealth of information accessible from any device. However, any individual user will want to format the information to suit his or her needs.  The SDF has a [`from_layer`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.SpatialDataFrame.from_layer) method that allows for consuming a `feature layer`, reporting data from it, and manipulating the data to a form that's comfortable and makes sense for the intended purpose.\n",
+    "[`Feature layers`](https://doc.arcgis.com/en/arcgis-online/share-maps/hosted-web-layers.htm) hosted on [**ArcGIS Online**](https://www.arcgis.com) or [**ArcGIS Enterprise**](http://enterprise.arcgis.com/en/) can be easily read into a Spatial DataFrame using the  [`from_layer`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.SpatialDataFrame.from_layer) method. Once you read it into a SDF object, you can create reports, manipulate the data to a form that's comfortable and makes sense for the intended purpose.\n",
     "\n",
     "**Example: Retrieving an ArcGIS Online [`item`](https://developers.arcgis.com/rest/users-groups-and-items/publish-item.htm) and using the [`layers`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.toc.html#layer) property to inspect the first 5 records of the layer**"
    ]
@@ -568,8 +567,10 @@
    "source": [
     "# Obtain the first feature layer from the item\n",
     "fl = known_item.layers[0]\n",
+    "\n",
     "# Use the `from_layer` method of the Spatial DataFrame to create a new Spatial DataFrame\n",
     "sdf = SpatialDataFrame.from_layer(fl)\n",
+    "\n",
     "# Return the first 5 records. \n",
     "sdf.head()"
    ]
@@ -585,7 +586,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "You can also use sql queries to return a subset of records by leveraging the ArcGIS API for Python's [`Feature Layer`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#featurelayer) object itself. Instantiate a `Pandas data frame` directly from the [`feature layer.query`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureLayer.query) method and use the data frame's [`head`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.GroupBy.head.html#pandas.core.groupby.GroupBy.head) method to return the first 5 records and a subset of columns from the DataFrame:\n",
+    "You can also use sql queries to return a subset of records by leveraging the ArcGIS API for Python's [`Feature Layer`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#featurelayer) object itself. Instantiate a Pandas `DataFrame` directly from the [`FeatureLayer.query()`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureLayer.query) method and use the data frame's [`head()`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.GroupBy.head.html#pandas.core.groupby.GroupBy.head) method to return the first 5 records and a subset of columns from the DataFrame:\n",
     "\n",
     "#### Example: Feature Layer Query Results to a Spatial DataFrame"
    ]
@@ -594,7 +595,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We'll use the `AGE_45_54` column to query the dataframe and return a new dataframe with a subset of records. We can use the built-in [`zip`](https://docs.python.org/3/library/functions.html#zip) function to print the dataframe atrribute field names in columns, and then use dataframe syntax to view specific attribute fields in the output:"
+    "We'll use the `AGE_45_54` column to query the dataframe and return a new `DataFrame` with a subset of records. We can use the built-in [`zip()`](https://docs.python.org/3/library/functions.html#zip) function to print the data frame attribute field names, and then use data frame syntax to view specific attribute fields in the output:"
    ]
   },
   {
@@ -603,7 +604,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Filter feature layer records with a sql query. See [`where clause documentation`](https://developers.arcgis.com/rest/services-reference/query-feature-service-layer-.htm).\n",
+    "# Filter feature layer records with a sql query. \n",
+    "# See https://developers.arcgis.com/rest/services-reference/query-feature-service-layer-.htm\n",
+    "\n",
     "df = fl.query(where=\"AGE_45_54 < 1500\").df"
    ]
   },
@@ -725,17 +728,17 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Accessing Feature Class Data\n",
+    "## Accessing local GIS data\n",
     "\n",
     "The SDF can also access local geospatial data. Depending upon what Python modules you have installed, you'll have access to a wide range of functionality:  \n",
     "\n",
-    "* If the **`ArcPy`** module is installed, meaning you have installed either [`ArcGIS Desktop`](http://desktop.arcgis.com/en/) or [`ArcGIS Pro`](http://pro.arcgis.com/en/pro-app/) (or both) and have installed the ArcGIS API for Python in that same environment, the Spatial DataFrame has methods to read a subset of the ArcGIS Desktop [supported geographic formats](http://desktop.arcgis.com/en/arcmap/10.3/manage-data/datatypes/about-geographic-data-formats.htm#ESRI_SECTION1_4835793C55C0439593A46FD5BC9E64B9), most notably:\n",
+    "* If the **`ArcPy`** module is installed, meaning you have installed [`ArcGIS Pro`](http://pro.arcgis.com/en/pro-app/) and have installed the ArcGIS API for Python in that same environment, the `SpatialDataFrame` has methods to read a subset of the ArcGIS Desktop [supported geographic formats](http://desktop.arcgis.com/en/arcmap/10.3/manage-data/datatypes/about-geographic-data-formats.htm#ESRI_SECTION1_4835793C55C0439593A46FD5BC9E64B9), most notably:\n",
     " * [`feature classes`](http://desktop.arcgis.com/en/arcmap/latest/manage-data/feature-classes/a-quick-tour-of-feature-classes.htm)\n",
     " * [`shapefiles`](http://desktop.arcgis.com/en/arcmap/latest/manage-data/shapefiles/what-is-a-shapefile.htm),  \n",
     " * [`ArcGIS Server Web Services`](https://enterprise.arcgis.com/en/server/latest/publish-services/windows/what-types-of-services-can-you-publish.htm) and [`ArcGIS Online Hosted Feature Layers`](https://doc.arcgis.com/en/arcgis-online/share-maps/publish-features.htm) \n",
     " * [`OGC Services`](http://www.opengeospatial.org/standards)  \n",
     "* If the **ArcPy** module is not installed, the SDF [`from_featureclass`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.SpatialDataFrame.from_featureclass) method only supports consuming an Esri [`shapefile`](http://desktop.arcgis.com/en/arcmap/latest/manage-data/shapefiles/what-is-a-shapefile.htm)\n",
-    "> Please note that you must install the `pyshp` module to read shapefiles with Python interpreters that don't have access to `ArcPy`.\n",
+    "> Please note that you must install the `pyshp` package to read shapefiles in environments that don't have access to `ArcPy`.\n",
     "    \n",
     "### Example: Reading a Shapefile\n",
     "> You must authenticate to `ArcGIS Online` or `ArcGIS Enterprise` to use the `from_featureclass()` method to read a shapefile with a Python interpreter that does not have access to `ArcPy` through either an ArcGIS Desktop or ArcGIS Pro installation.\n",
@@ -1002,25 +1005,6 @@
     "#### Example: Export a whole dataset to a shapefile:"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "> **Note:** You may receive an error message that ends with the line `SystemError: <built-in function isinstance> returned a result with an error set` the first time you run the `sdf.to_featureclass()` method. If you run the command again, the output reports the path to the feature class and the error message does not appear.  As a workaround, you can direct the output to your `temp` directory and retrieve the output (although the error message may still appear):"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Workaround to the `SystemError` warning message appearing upon first run of `sdf.to_featureclass()`\n",
-    "import tempfile\n",
-    "folder = tempfile.gettempdir()\n",
-    "sdf.to_featureclass(out_location=folder, out_name=\"my_export2.shp\")"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 13,
@@ -1039,14 +1023,14 @@
    ],
    "source": [
     "sdf.to_featureclass(out_location=r\"path\\to\\your\\data\\output_example\",\n",
-    "                   out_name=\"output_cities.shp\")\n"
+    "                   out_name=\"output_cities.shp\")"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "> The ArcGIS API for Python on all `macOS` and `Linux` machines, as well as those `Windows` machines not using Python interpreters that access ArcGIS Desktop or ArcGIS Pro installations will only be able to write out to shapefile format with the `to_featureclass` method. Writing to file geodatabases requires the `ArcPy` site-package."
+    "> The ArcGIS API for Python installs on all `macOS` and `Linux` machines, as well as those `Windows` machines not using Python interpreters that have access to `ArcPy` will only be able to write out to shapefile format with the `to_featureclass` method. Writing to file geodatabases requires the `ArcPy` site-package."
    ]
   },
   {
@@ -1124,7 +1108,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.4"
   }
  },
  "nbformat": 4,

guide/05-working-with-the-spatial-dataframe/visualizing-data-with-the-spatial-dataframe.ipynb

@@ -9,28 +9,21 @@
     "\n",
     "Some unique characteristics of working with the visualization capabalities on the SDF:\n",
     "- Uses Pythonic syntax\n",
+    "- Uses the same syntax as visualizing charts on Pandas DataFrames\n",
     "- Uses symbology familiar to users of matplotlib\n",
     "- Works on features and attributes simultaneously, eliminating to a great extent the need to iterate over all features (rows)\n",
     "- Handles reading and writing to multiple formats aiding data conversion\n",
     "\n",
-    "Dataset symbology can be customized on a `Spatial DataFrame` or a `Pandas data frame`. You can easily create dataframes from the results of a [`FeatureLayer.query()`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureLayer.query) operation. The resulting [`FeaturSet`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureSet) object has a [`df`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureSet.df) method from which you can visualize features. \n",
+    "Dataset symbology can be customized on a `SpatialDataFrame` or a Pandas `DataFrame`. You can easily create `DataFrame`s from the results of a [`FeatureLayer.query()`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureLayer.query) operation. The resulting [`FeaturSet`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureSet) object has a [`df`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.FeatureSet.df) property from which you can visualize features.\n",
     "\n",
-    "* [Visualizing Spatial Data](#Visualizing-Spatial-Data)\n",
-    " * [Plotting the dataframe geometry](#Plotting-the-dataframe-geometry)\n",
+    "In this guide, you will learn about:\n",
+    "\n",
+    " * [Plotting the SpatialDataFrame](#Plotting-the-SpatialDataFrame)\n",
     " * [Color Maps and Colors](#Color-Maps-and-Colors)\n",
     " * [Renderers](#Renderers)\n",
     " * [Symbology for Simple Renderers](#Symbology-for-Simple-Renderers)"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from arcgis.gis import GIS"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 9,
@@ -252,12 +245,25 @@
    ],
    "source": [
     "from arcgis import GIS\n",
+    "\n",
+    "# create an anonymous connection to ArcGIS Online and get a public item\n",
     "item = GIS().content.get(\"85d0ca4ea1ca4b9abf0c51b9bd34de2e\")\n",
     "flayer = item.layers[0]\n",
+    "\n",
+    "# Specify a SQL query and get a SpatialDataFrame object\n",
     "df = flayer.query(where=\"AGE_45_54 < 1500\").df\n",
+    "\n",
+    "# Visualize the top 5 records\n",
     "df.head()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> Note: If the above cell looks new to you, please checkout the [Introduction to Spatial DataFrame](/python/guide/features-module/introduction-to-spatial-dataframe) guide."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -297,7 +303,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Plotting the dataframe geometry\n",
+    "### Plotting the SpatialDataFrame\n",
     "The code below plots a set of points on the map above using a common structure used amongst many different Python packages for defining symbology. It is built off of the matplotlib libraries for simple, straightforward plotting. We'll explain some of the parameters below, and the [`plot() API Reference`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html#arcgis.features.SpatialDataFrame.plot) outlines more options. \n",
     "\n",
     "See also the [`matplotlib.pyplot`](https://matplotlib.org/api/pyplot_summary.html) or Pandas dataframe [`plot`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.plot.html) documentation for further details."
@@ -321,9 +327,9 @@
    ],
    "source": [
     "df.plot(kind='map', \n",
-    "        map_widget=m,\n",
+    "        map_widget=m, # optional - you can specify an existing map or ignore to create a new map\n",
     "        symbol_type='simple',\n",
-    "        symbol_style='d',\n",
+    "        symbol_style='d', # d - for diamonds\n",
     "        colors='Reds_r',\n",
     "        cstep=10,\n",
     "        outline_color='Blues',\n",
@@ -593,6 +599,7 @@
     }
    ],
    "source": [
+    "# get the styles that are relevant to the current geometry type (points)\n",
     "show_styles(df.geometry_type)"
    ]
   },
@@ -674,7 +681,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.4"
   }
  },
  "nbformat": 4,