Add json endpoints for open data policy compliance.

[jj0hns0n]

One thing that might be good to add to the feature wish list (at least for federal customers) is to have GeoNode satisfy the new 'open data' policy for its own content. That would mean providing their JSON output format at geonode.example.com/data.

http://project-open-data.github.io/schema/
http://project-open-data.github.io/examples/catalog-sample-extended.json

[pjdufour]

See the following page to learn how to get geospatial data into Data.Gov and GeoPlatform.

http://www.digitalgov.gov/resources/how-to-get-your-open-data-on-data-gov/#federal-geospatial-data

The CSW endpoint is still accepted (and I would say preferred to data.json). For many agencies the data.json endpoint is controlled by a non-geospatial office and is managed in a legacy system (SharePoint, or manually written, etc.), which creates a significant bottleneck. A /data.json endpoint on GeoNode would not be used.

To help agencies fulfill their OMB requirements, fixing metadata and CSW bugs associated with CKAN integration (https://github.com/GeoNode/geonode/labels/ckan-integration) will enable direct harvest from GeoNode into data.gov and agency level CKANs.

[tomkralidis]

@kalxas has implemented the data.gov CSW and could perhaps speak to CKAN harvesting. It's worth articulating the gaps in GeoNode's metadata model on CSW output so we can apply and achieve harvesting/federation.

[pjdufour]

Yes, hopefully we can do a more thorough review of the gaps at some point (Comparing ISO 19115 vs. actual GeoNode implementation vs. OMB/data.json requirements vs. CKAN implementation).

CC @JJediny

[kalxas]

Hi,
Harvesting from CSW is the best option in my opinion (large US federal organizations are deploying pycsw to cover this requirement), so we need to fix all CSW related issues.
Of course It would be great to have OpenDataJSON support.
For data.gov we implemented a harvester from OpenDataJSON to ISO 19115 in order to get all those records in pycsw.
How about an OpenDataJSON output format plugin for pycsw to do the reverse?

[mwengren]

All,

FWIW, I did some testing before FOSS4G last year on GeoNode CSW -> CKAN harvesting via CS-W. The shortcomings that I noticed on the CKAN side were:

1. would not present a WMS 'Preview Map' link without a full GetCapabilties URL present within GeoNode CS-W/ISO 199115 OnlineResource links
2. parsing of GeoServer-related links and content in CKAN's ckanext-spatial extension was sub-par and wouldn't always identify a WFS, WMS, etc resource properly.

For # 2, I wrote some code in a ckanext-spatial branch (here: https://github.com/mwengren/ckanext-spatial/tree/resource_format_detection_fix), to begin to fix some of these by using regex pattern matches in the CKAN URL parsing code, and also to parse the gmd:OnlineResource/gmd:protocol ISO elements for service type identifiers found in @tomkralidis's Cat-Interop repo: https://github.com/OSGeo/Cat-Interop in favor of parsing URLs, where available (GeoNode includes these in the appropriate places). Still needs some work and testing before being merged into the main CKAN extension, but we are currently using this on our CKAN-based NOAA Data Catalog: https://data.noaa.gov.

For # 1, as a workaround I just modified GeoNode to serve back a full GetCaps link, but in addition to #2 both issues are probably better addressed on the CKAN side in terms of code modifications, if we agree that a full GetCaps link isn't necessary in GeoNode CS-W output.

Apart from any other issues already noted in GitHub, with these changes the results are pretty decent in terms of GeoNode CS-W harvest by CKAN, for data layers anyway. Just wanted to report what I know on the topic in case it's helpful.

Micah