References

General Resources and References
Gridded Ocean Data Available for Visualization and Download Policy Documents/References
Tutorials and References Ontologies
Notebooks By Category
Abbreviated Terms

## General Resources and References * [The Open Geospatial Consortium. Open Geospatial Consortium Reference Model](www.opengis.net/doc/orm/2.1.) * [Catalog Services for the Web Features](http://docs.geoserver.org/latest/en/user/extensions/csw/features.html) * Provides operations supported, information on supported schemes, mapping files, * [OpenSearch](http://www.opensearch.org/Home): A collection of simple formats for sharing of search results * [GEOSS AIP-6 Architecture Document](http://www.earthobservations.org/documents/cfp/201302_geoss_cfp_aip6_architecture.pdf) * [GEOSS AIP-3 Use Cases Engineering Report GEOSS Architecture Implementation Pilot, Phase3,](http://www.ogcnetwork.net/pub/ogcnetwork/GEOSS/AIP3/documents/AIP-3_Use_Cases_ER110210.pdf) * [Paper on DMAC technologies in support of modeling](http://www.mdpi.com/2077-1312/2/1/194) * * Karl Benedicts presentation to the DMAC ST on 2013-11-19. * Links to the DATE project presentation materials for tomorrow morning's session: * [Presentation (self-contained web page. arrow-keys advance)](https://www.dropbox.com/s/ehlu0p3s7caxp8i/presentation.slides.html) * [Notes (just an alternative format for the same content)](https://www.dropbox.com/s/0spmxekonaivneu/presentation.html) * [Rich Signell's presentation at the same meeting](https://drive.google.com/file/d/0BzAHlPEEP_ujTHo2MHN0akdPLTA/edit?usp=sharing) * [Example IPython Notebook Demonstrating Search, Access](https://www.wakari.io/sharing/bundle/rsignell/NGDC-CSW-DAP) * Rich's IPython Tutorial](https://www.youtube.com/watch?v=4NyMWK4as-U) * Invasive Species Data Cataloging Methods * [Global Invasive Species Information Network](http://www.gisin.org/DH.php?WC=/WS/GISIN/GISINDirectory/WebservicesForConsumers.html&WebSiteID=4) * [USGS Documenting, Mapping, and Predicting Invasive Species Model](http://www.fort.usgs.gov/Research/research_tasks.asp?TaskID=2190) * [USDA Aquatic Species Databases](http://www.invasivespeciesinfo.gov/aquatics/databases.shtml) * Sea Level Rise and Inundation Case Studies * [Elevated East Coast Sea Level Anomaly, NOAA Technical Report NOS CO-OPS 051](http://tidesandcurrents.noaa.gov/publications/EastCoastSeaLevelAnomaly_2009.pdf) * [Gridded Ocean Data Available for Visualization and Download](http://blog.marinexplore.com/15-beautiful-ocean-data-visualizations/) * A number of visualization models are linked from the Marineexplore.com blog ### Policy and Standards Documents * [NetCDF CF Metadata Conventions](http://cfconventions.org) * [IOOS Build Out Plan Synthesis](http://www.ioosassociation.org/sites/nfra/files/documents/ioos_documents/regional/BOP%20Synthesis%20Final.pdf) * [FEMA report on post Sandy](http://www.fema.gov/media-library/assets/documents/33772) * [Strategic Plan for Federal Research and Monitoring of Ocean Acidification](http://www.whitehouse.gov/sites/default/files/microsites/ostp/NSTC/iwg-oa_strategic_plan_march_2014.pdf) * See Theme 2. Monitoring of Ocean Chemistry and Biological Impacts (P 28) * [NMFS Data Documentation Procedural Directive](https://inport.nmfs.noaa.gov/inport/downloads/data-documentation-procedural-directive.pdf) * [Dublin Core Metadata Initiative](http://dublincore.org/documents/dces/) * [ISO Standards](http://www.iso.org/iso/home/standards.htm) * [NISO Standards](http://www.niso.org/apps/group_public/download.php/10256/Z39-85-2012_dublin_core.pdf) ### Tutorials and References * [Editor Commands](http://www.cs.rit.edu/~cslab/vi.html) * Exploring Catalogs * [IOOS Service Registries Github](https://github.com/ioos/registry) * This site also provides informative webinars about how data is harvested from the regions. * [Step By Step Approach to Exploring Web Map Services](http://nbviewer.ipython.org/github/Unidata/tds-python-workshop/blob/master/wms_sample.ipynb) * This Notebook is really valuable for looking at code to access various aspects of a map layer and print out available data on those aspect and is written as a tutorial. * [WMS Documentation](http://mapserver.org/ogc/wms_server.html#changing-the-online-resource-url) * Wakari Tutorials * [Versioning Wakari notebooks on Github](https://www.youtube.com/watch?v=0SQa2a18tMM) * [IPython Notebook Documents](http://ipython.org/ipython-doc/dev/interactive/notebook.html) * [Tutorial on Wakari Environments](https://www.youtube.com/watch?v=6mxCf8a_rMM) * [Anaconda Environments How To](https://www.wakari.io/docs/anaconda.html) * [Custom Wakari Environments](http://continuum.io/blog/wakari_custom_envs) * [owslib Installation Instructions](https://binstar.org/rsignell/owslib/0.8.6) * [OWSLib 0.8.6 documentation](http://geopython.github.io/OWSLib/) * [IPython Tutorial on Accessing Geospatial Data, including NASA data](http://nbviewer.ipython.org/github/profLewis/geogg122/blob/master/Chapter4_GDAL/GDAL_HDF.ipynb) * [Install Instructions for Wakari Environment](https://github.com/esc24/installation-recipes/blob/78e4b70345c0d093bc2a68f747ad464f584f648e/wakari/install.txt) * [pyoos](https://github.com/asascience-open/pyoos) * [ulmo](http://ulmo.readthedocs.org/en/latest/) sites and documentation * Python Tutorials * [The Python Standard Library](http://docs.python.org/2/library/) * [Python Lists and Loops](http://www.icrar.org/__data/assets/pdf_file/0008/1436615/challenge09b-notes3.pdf) * [Python Dicts](https://developers.google.com/edu/python/dict-files) * [Library of Python Packages](https://github.com/asascience-open) * [Python Tutorial in Ipython notebooks](http://nbviewer.ipython.org/github/profLewis/geogg122/blob/master/Chapter2_Python_intro/python101.ipynb) * [matplotlib Overview and User's Guides](http://matplotlib.org/contents.html) * [pycsw Workshop Materials](http://www.coastalmarinedata.net/pycsw-workshop-materials-now-posted/) * [Shell Commands](http://learncodethehardway.org/cli/book/cli-crash-course.html#introduction-shut-up-and-shell) * Geospatial Tools In Python * [Notebook on Geospatial analysis in Python including Tutorial Video](http://nbviewer.ipython.org/github/mqlaql/geospatial-data/blob/master/Geospatial-Data-with-Python.ipynb) * [Chapter4_GDAL/OGR_Python](http://nbviewer.ipython.org/github/profLewis/geogg122/blob/master/Chapter4_GDAL/OGR_Python.ipynb) * [Bounding Box Documentation](http://rk700.github.io/python-fitz/bbox.html) * [Shapely User Manual](http://toblerity.org/shapely/manual.html) * [Useful Bounding Box Finder](http://bboxfinder.com/) * Miscelaneous * [Online Stats Reference](https://www.statsoft.com/Textbook/Elementary-Statistics-Concepts/button/1) * [KML Documentation](https://developers.google.com/kml/documentation/kml_tut) * [Instructions for building my Wakari environment](https://github.com/ioos/ipython-notebooks/blob/master/creating_wakari_environments.md), and the [Wakari notebook that bundles the environment (so you can install it easily)](https://www.wakari.io/sharing/bundle/emayorga/pyoos_ioos_sos_demo1) ## Ontologies * [Marine Metadata Interoperability Ontology Registry and Repository/Biological Terms](http://mmisw.org/orr/#http://mmisw.org/ont/ioos/biological) * [Additional MMI Endpoints for "Biological" Terms](http://mmisw.org/orr/#b) * [NOAA ERDDAP](http://coastwatch.pfeg.noaa.gov/erddap/index.html) * [Setting up ERDDAP](http://coastwatch.pfeg.noaa.gov/erddap/download/setup.html) * [NOAA ERDDAP API](http://www.programmableweb.com/api/noaa-erddap) * Regional ERDDAP Data Set Searches * [PACIOOS ERDDAP](http://oos.soest.hawaii.edu/erddap/info/index.html?page=1&itemsPerPage=1000) * [NERACOOS ERDDAP](http://www.neracoos.org/erddap/index.html) ## Existing Notebooks By Category

• Collections of Notebooks by emayorga
• Collections of Notebooks by rsignell

###NetCDF4 files --> (NetCDF simply provides a way of opening datasets as files or remote access URLs and offers a library of functions to use on those datasets)

• National Data Buoy Center
  • examining groups, variables, and time series plot using pandas
• ADCIRC / [Model Testing] (https://www.wakari.io/sharing/bundle/rsignell/ADCIRC_water_level_and_velocity_3d)
  • using NetCDF4-Python library to access velocity data from a triangular grid ocean model via OPeNDAP, specifying the desired URL, time, layer and lat/lon region of interest
• Access data in Wakari from the OPeNDAP Web Service using NetCDF4-Python
  • using NetCDF4-Python, pandas, creating plot
• Testing CF Conventions on an SJROFS Ocean Forecast Model
  • using CF Conventions in the context of St. John's River Operational Forecast System (SJROFS)
• Plotting Argo Data and Glider Data from Australia's Integrated Marine Observing System (IMOS)
  • using netCDF, python, matplotlib
• Reading NCEP ADP Global Upper Air and Surface Weather Observations (PREPBUFR format)
  • uses netCDF, pandas (including writing to csv)
• Examining Longitude and Latitude Extent in Directory Access Protocol endpoint (GLOS)
• Extract and plot data from a NetCDF file or OPeNDAP dataset
• Using pyugrid to examine netCDF4 Dataset
  • uses pyugrid, a Python API to utilize data written using the unstructured grid UGRID conventions, as well as matplotlib.tri to plot a contour map

###ERDDAP

• Examining and Plotting GTS Data from ERDDAP
  • Using REST Requests, pandas, read_csv

###SciTool Python Packages for Cartography

• Examining a Regional Ocean Model Using Iris
  • uses Iris
• Examining Cartopy Visualization Tools
  • uses Cartopy
• Using Iris to Access Data from U.S. IOOS Regional and Federal Models
  • uses Iris, time_near, var_lev_date, myplot, USGS/COASWST, MARACOOS/ESPRESSO, SECOORA/NCSU, CENCOOS/UCSC, HIOOS, Global RTOFS/NCEP
  • See Also, Using Iris to Access Data from U.S.-IOOS models which builds on notebook by accessing models via CSW, rather than specifying urls
    • this additional notebook is valuable in that it demonstrates how to use a CSW endpoint (e.g. NGDC/IOOS Geoportal, or NODC.UAF Geoportal) and then using PropertyIsLike and a text search to discover available models
  • See Also, Using Iris to Access IOOS Models, identifying Iris Bug

###Testing IOOS/Regional Services

• Ingesting USGS Data with ulmo via CUAHSI HIS Central
• Accessing CUAHSI HIS Central "discovery" web services
• Using Pyoos to access Axiom 52North IOOS SOS "Stable Demo"
• Access the operational NDBC SOS service with two methods: Pyoos NDBC SOS collector, and OWSLib/SOS
• Ingesting USGS data with ulmo via CUAHSI HIS Central
• Use OWSLib to examine a DescribeSensor/SensorML response from IOOS SOS Milestone 1 (AOOS 52N SOS), then check vocabulary terms against MMI IOOS vocabularies
• IOOS Models displayed in Iris and using CSW to query (with environment)
• IOOS 52 North SOS / Pyoos Testing Notebook
  • uses pyoos, a Python library for collecting Met/Ocean observations, to test and examine a particular IOOS service
• Accessing IOOS Coastal Ocean Modeling Testbed (COMT)
  • examining Western North Atlantic, Caribbean and Gulf of Mexico Tidal Databases
  • using netCDF4, pandas, matplotlib, pylab, date time to plot water depth against water velocity vectors on a map and create a time series of water levels at a particular location
• Accessing Northeastern Coastal Ocean Forecast System using the NetCDF4-Python
  • examining NECOFS
  • using netCDF4, pandas, read_csv to examine water level at particular stations according to the model by finding (x,y) coordinates near desired station locations and then plotting these water levels on a time series graph
• Accessing an IOOS approved web service using python
  • using pandas, cStringIO, pyoos, read_csv and CoopsSos
• Accessing New England Coastal Ocean Forecast System via OPeNDAP
  • using pylab, matplotlib.tri, netCDF4
  • examining the New England Coastal Ocean Forecast System and THREDDS Data Server
• Extracting time series from NWS NOAA/NCEP Wavewatch III
  • using NWS NOAA/NCEP Wavewatch III, pandas, datetime
  • provides an example of extracting and plotting a 3D time series
• Access CSW to determine ServiceType
  • accessing NODC, NGDC, and DATA.NOAA.GOV, using CSW, netCDF4 to access model data
• Accessing CSW with OWSLib using ISO queryables
  • using pylab, owslib.csw, netCDF4, pandas, datetime
• Using regional data and independent CSV data to approach marine Conflict Resolution Planning
  • uses pandas, netCDF4, read_csv, group.to_csv, and demonstrating how to use matplotlib to create histograms of the data and then group the data by month on maps of bathymetry
  • this notebook was created as an approach to resolving conflicts between aquaculture siting and Right Whale protection in Cape Cod Bay
• Accessing IOOS Coastal Ocean Modeling Testbed (COMT)
  • using COMT, CSW, pandas, matplotlib.tri, netCDF4, datetime, pyugrid
  • demonstrates finding nearby points of interest from a model and harvesting service url's from a CSW endpoint
  • demonstrates time series plotting different outputs of water levels from a plot over a certain time period
• Accessing Data from a NOAA Forecast Model via OPeNDAP
  • using San Francisco Bay Operational Forecast System (SFBOFS)
  • demonstrating use of netCDF4, matplotlib.try, pylab, date time
  • visualizing a tricontourf plot of water depth with water current vectors overlaid
• Visualizing an unstructured grid Model output from the IOOS Modeling Testbed
  • using netCDF4 to access OPeNDAP using Unidata NetCDF-C Library
  • [Unstructured Grids] (https://publicwiki.deltares.nl/display/NETCDF/Unstructured+grids) are a more primitive form of data set that stores topology and point coordinates explicitly, meaning that information about relationship to other polygons is part of the data, and the data takes up more memory. In contrast, structured grids store information implicitly, meaning that they tag information with an ID that is based on the position of the information in a data file.
    • Additional Resources: VTK Data Model for visualizations of unstructured vs. structured grid data; IOOS DIF Model Data Interoperability
  • UGRID Interoperability Forum
• [Assessing OWSLIB SOS] (https://www.wakari.io/sharing/bundle/emayorga/OWSLibSOSAccess_Test1)
  • This notebook may be valuable as it does not use pyoos, which seems to require the development of a custom environment in wakari.

###Comparing Data Catalogs

• Comparing NOAA Geoportal to Data.Gov
  • using CSW to compare NODC geoportal to data.gov
• Comparing model results from NODC and NGDC
  • using CSW, ServiceType, OPenDAP
• Accessing CSW metadata for particular variable, bounding box and visualization
  • using pylab, owslib.csw, netCDF4, pandas, datetime
  • includes list of CSWs
• Accessing ncSOS with OWSLib and Pyoos
  • using NcSOS, which adds an OGC SOS service to datasets in THREDDS server and complies with IOOS SWE Milestone 1.0 templates
  • using pandas, owlsib.etree, pdb, oslib.sos, datetime
• Using CSW to search for datasets with a certain variable in a certain bounding box and determining Service endpoints
  • using ISO Queryables (these are incredibly valuable in terms of determining a set of records that can be accessed through particular Services (WCS, WMS, WFS, SOS)), netCDF4, pandas
  • See also NGDC CSW Notebook using the same methods
• Accessing data.gov using CKAN API
  • utilizing ckanclient, and pandas
• Accessing data.noaa.gov to find data on a certain variable, in a certain bounding box, and with a certain data endpoint
  • using pandas, CKCAN API, data.noaa.gov API
  • See also, Open Standards and Interoperability
  • this notebook is valuable in that it demonstrates how to retrieve urls and then examine the service endpoints for a particular variable within a set geographic area
• Accessing data.gov using CKAN API
  • provides a detailed narrative using CKAN API with requests package and ckanclient package
  • also demonstrates how to print out url resources for each record and dataset service endpoints by looking for NetCDF
  • narrative is an example of taking a look at data sets and pointing out lack of meta data that makes it hard to utilize the data

###Miscellaneous

• Package for creating IPython Blog Environment

## Abbreviated Terms **API** Application Program Interface **DCP** Distributed Computing Platform **GML** Geographic Markup Language **ISO** International Organization for Standardization **OGC** Open Geospatial Consortium **OWS** OGC Web Services **O&M** Observations and Measurements **SensorML** Sensor Model Language **SOS** Sensor Observation Service [formerly Sensor Collection Service, SCS] **SPS** Sensor Planning Service **SWE** Sensor Web Enablement **TML** Transducer Markup Language **UML** Unified Modeling Language **WNS** Web Notification Service **XML** eXtended Markup Language