Skip to content

SolarUser Datum Export API

Jump to bottom Edit New page
Matt Magoffin edited this page Jan 31, 2019 · 13 revisions

The SolarUser Datum Export API provides methods to manage data export tasks. All paths are relative to a /solaruser prefix. All dates and times are represented in the Gregorian calendar system. All requests must provide a valid authentication token. See SolarNet API authentication for information on how to use authentication tokens.

Endpoints

Verb Endpoint Description
GET /user/export/adhoc List previously submitted ad hoc export tasks.
POST /user/export/adhoc Submit an ad hoc export task request.
GET /user/export/services/compression List the available export compression types.
GET /user/export/services/destination List the available export destination types.
GET /user/export/services/output List the available export output format types.

Localized responses

Many endpoints return localized messages. The locale of the response is determined by the Accept-Language HTTP header passed on the request. If not provided, the default locale of the SolarNetwork service will be used.

Localized setting specifiers

Some responses include a localizedInfoMessages response object. This objects contains a pair of localized messages for the setting specifier key values returned in the same response. The localized message pairs are in the form X.key (a title) and X.desc (a longer description) where X is a setting specifier key. For example, a response might look like:

{
  "id": "net.solarnetwork.central.datum.biz.impl.CsvDatumExportOutputFormatService",
  "locale": "en-NZ",
  "settingSpecifiers": [
	{
	  "key": "includeHeader",
	  "defaultValue": true,
	  "trueValue": true,
	  "falseValue": false,
	  "transient": false,
	  "type": "net.solarnetwork.settings.ToggleSettingSpecifier"
	}
  ],
  "localizedDescription": "Export data in comma separated values (spreadsheet) format.",
  "localizedInfoMessages": {
	"includeHeader.key": "Include Header",
	"includeHeader.desc": "Toggle the inclusion of a CSV header row."
  },
  "localizedName": "CSV"
}

A single includeHeader setting is defined for this service. The localizedInfoMessages object thus defines the title for that setting via includeHeader.key (Include Header) and a description via includeHeader.desc (Toggle the inclusion of a CSV header row.).

List ad hoc export tasks

List the available ad hoc export tasks previously submitted via a POST to this same URL.

GET /api/v1/sec/user/export/adhoc
states An optional comma-delimited list of task state key values. If not provided all states are returned.
success An optional true or false to only include tasks with a matching success value. If not provided all success values are returned.

For example, to find all pending tasks you'd call this API like:

/api/v1/sec/user/export/adhoc?states=q,p,e

To find all failed tasks you'd call this API like:

/api/v1/sec/user/export/adhoc?states=c&success=false

To find all completed tasks (both successful and failed) you'd call this API like:

/api/v1/sec/user/export/adhoc?states=c

List ad hoc export tasks response

The response contains a list of ac hoc export tasks. Tasks have the following properties:

Property Type Description
userId number The ID of the user that owns the task.
exportDate string The task export date, in YYYY-MM-dd HH:mm:ss.SSS'Z' format, in the UTC time zone.
scheduleTypeKey string Will always be the value a for ad hoc.
created string The date the task was submitted, in YYYY-MM-dd HH:mm:ss.SSS'Z' format, in the UTC time zone.
taskId string A unique identifier assigned by SolarNetwork.
config object The export configuration object.
task object An export task execution object.

An example response looks like:

{
  "success": true,
  "data": [
    {
      "userId": 7,
      "exportDate": "2019-01-31 21:36:33.723Z",
      "scheduleTypeKey": "a",
      "created": "2019-01-31 21:36:33.750Z",
      "taskId": "0968234c-0c6e-4a14-8e7b-9148f7e41080",
      "config": {
        "name": "Ad hoc: 2019-01-31T21:36:33+00:00",
        "dataConfiguration": {
          "datumFilter": {
            "nodeIds": [
              11
            ],
            "sourceIds": [
              "Main"
            ],
            "aggregation": "None",
            "aggregationKey": "0",
            "mostRecent": false,
            "startDate": "2014-03-01 08:00:00.000Z",
            "endDate": "2014-03-31 11:00:00.000Z",
            "offset": 0,
            "location": {},
            "withoutTotalResultsCount": false
          }
        },
        "outputConfiguration": {
          "serviceIdentifier": "net.solarnetwork.central.datum.export.standard.CsvDatumExportOutputFormatService",
          "compressionTypeKey": "x",
          "serviceProperties": {
            "includeHeader": true
          }
        },
        "destinationConfiguration": {
          "serviceIdentifier": "net.solarnetwork.central.datum.export.dest.s3.S3DatumExportDestinationService",
          "serviceProperties": {
            "path": "s3-us-west-2.amazonaws.com/solarnetwork-dev-testing/data-exports",
            "accessKey": "mRWO6qzn57AFfIq0Qo60zyD3",
            "secretKey": "{SSHA-256}CXZ3Rz5botMSZi4AOyKgxoJr8OoCw==",
            "filenameTemplate": "adhoc-data-export-{date}.{ext}"
          }
        },
        "schedule": "Adhoc",
        "hourDelayOffset": 0
      },
      "task": {
        "id": "0968234c-0c6e-4a14-8e7b-9148f7e41080",
        "created": "2019-01-31 21:36:33.750Z",
        "modified": "2019-01-31 21:41:11.925Z",
        "exportDate": "2019-01-31 21:36:33.723Z",
        "config": {
          ...
        },
        "statusKey": "c",
        "completionDate": "2019-01-31 21:41:11.923Z",
        "success": true
      }
    }
  ]
}

Submit ad hoc export task

Submit an ad hoc export task configuration for asynchronous execution. Once submitted, the export task will be executed at some point in the future. Use the list endpoint to check on the status of each task.

The request must be submitted as application/json as a JSON task configuration object with the following properties:

Property Type Description
name string A name for the export task.
dataConfiguration object Configuration object that defines which data to export.
outputConfiguration object Configuration object that defines the format to encode the data as.
destinationConfiguration object Configuration object that defines where to export the data to.

For example, the following task will export node 123, source /Meter/1 raw data for all of 2011 (UTC) as CSV data with a header row, compressed using xz, to S3 at /my-bucket/data-exports:

{
  "name": "Ad hoc: 2019-01-31T21:36:33+00:00",
  "dataConfiguration": {
    "datumFilter": {
      "nodeIds": [
        123
      ],
      "sourceIds": [
        "/Meter/1"
      ],
      "startDate": 1293840000000,
      "endDate": 1325376000000
    }
  },
  "outputConfiguration": {
    "compressionTypeKey": "x",
    "serviceIdentifier": "net.solarnetwork.central.datum.export.standard.CsvDatumExportOutputFormatService",
    "serviceProperties": {
      "includeHeader": true
    }
  },
  "destinationConfiguration": {
    "serviceIdentifier": "net.solarnetwork.central.datum.export.dest.s3.S3DatumExportDestinationService",
    "serviceProperties": {
      "path": "s3-us-west-2.amazonaws.com/my-bucket/data-exports",
      "filenameTemplate": "adhoc-data-export-{date}.{ext}",
      "accessKey": "mRWO6qzn57AFfIq0Qo60zyD3",
      "secretKey": "alkjdsfoaseifuasldfasdfas"
    }
  }
}

Submit ad hoc export tasks response

The response will be a task object. See the list response for more details.

List compression types

This method will list the compression types supported by the SolarNetwork datum export service.

GET /api/v1/sec/user/export/services/compression

List compression types response

The response contains an array of compression service definitions.

Property Type Description
id string The service unique identifier.
settingSpecifiers array The configurable setting specifiers for the service.
locale string The locale of the localized messages in the response.
localizedName string A localized name for the service.
localizedDescription string A localized description of the service.
localizedInfoMessages object Localized messages for the associated setting specifiers.

An example response looks like:

{
  "success": true,
  "data": [
    {
      "id": "n",
      "locale": "en-NZ",
      "localizedDescription": "No compression.",
      "localizedName": "None"
    },
    {
      "id": "g",
      "locale": "en-NZ",
      "localizedDescription": "GZIP compression.",
      "localizedName": "GZIP"
    },
    {
      "id": "x",
      "locale": "en-NZ",
      "localizedDescription": "XZ compression.",
      "localizedName": "XZ"
    }
  ]
}

List destinations

This method will list the destinations supported by the SolarNetwork datum export service.

GET /api/v1/sec/user/export/services/destination

SolarNetwork supports the following destinations:

Destination Description
S3 Upload the export result to any AWS S3 compatible server.

S3 destination

The S3 destination uploads the export result to AWS S3 (or any compatible service). It supports the following settings:

Setting Type Description
path string The full path to the S3 endpoint, bucket, and folder to use. For example s3-ap-southeast-2.amazonaws.com/my-bucket/folder.
filenameTemplate string A template filename to use. This template is allowed to contain parameters in the form {key}, which are replaced at runtime by the value of a parameter key, or an empty string if no such parameter exists. The supported parameters are {date} and {ext} (file extension).
accessKey string The AWS access key to use for uploading the exported data.
secretKey string The AWS secret key to use for uploading the exported data.

List destinations response

The response contains an array of destination service definitions.

Property Type Description
id string The service unique identifier.
settingSpecifiers array The configurable setting specifiers for the service.
locale string The locale of the localized messages in the response.
localizedName string A localized name for the service.
localizedDescription string A localized description of the service.
localizedInfoMessages object Localized messages for the associated setting specifiers.

An example response looks like:

{
  "success": true,
  "data": [
    {
      "id": "net.solarnetwork.central.datum.export.dest.s3.S3DatumExportDestinationService",
      "locale": "en-NZ",
      "settingSpecifiers": [
        {
          "key": "path",
          "defaultValue": "",
          "secureTextEntry": false,
          "transient": false,
          "type": "net.solarnetwork.settings.TextFieldSettingSpecifier"
        },
        {
          "key": "filenameTemplate",
          "defaultValue": "data-export-{date}.{ext}",
          "secureTextEntry": false,
          "transient": false,
          "type": "net.solarnetwork.settings.TextFieldSettingSpecifier"
        },
        {
          "key": "accessKey",
          "defaultValue": "",
          "secureTextEntry": false,
          "transient": false,
          "type": "net.solarnetwork.settings.TextFieldSettingSpecifier"
        },
        {
          "key": "secretKey",
          "defaultValue": "",
          "secureTextEntry": true,
          "transient": false,
          "type": "net.solarnetwork.settings.TextFieldSettingSpecifier"
        }
      ],
      "localizedDescription": "Save exported data to AWS S3.",
      "localizedInfoMessages": {
        "path.key": "S3 Path",
        "path.desc": "The full path to the S3 endpoint, bucket, and folder to use. For example <code>s3-ap-southeast-2.amazonaws.com/my-bucket/folder</code>.",
        "filenameTemplate.key": "Filename",
        "filenameTemplate.desc": "A template filename to use. This template is allowed to contain parameters in the form <code>{key}</code>, which are replaced at runtime by the value of a parameter <code>key</code>, or an empty string if no such parameter exists. The supported parameters are <code>{date}</code> and <code>{ext}</code> (file extension).",
        "accessKey.key": "Access Key",
        "accessKey.desc": "The AWS access key to use for uploading the exported data.",
        "secretKey.key": "Secret Key",
        "secretKey.desc": "The AWS secret key to use for uploading the exported data."
      },
      "localizedName": "S3"
    }
  ]
}

List output formats

This method will list the output formats supported by the SolarNetwork datum export service. Output formats determine what the exported data will be encoded as.

GET /api/v1/sec/user/export/services/output

SolarNetwork supports the following output formats:

Destination Description
CSV Format the data as comma separated values (spreadsheet).
JSON Format the data as JSON.

CSV output format

ID net.solarnetwork.central.datum.export.standard.CsvDatumExportOutputFormatService

The CSV output format encodes each datum as a CSV row, with datum properties as columns. It supports the following settings:

Setting Type Description
includeHeader boolean Toggle the inclusion of a CSV header row. Defaults to true.

Example output looks like this:

created,nodeId,sourceId,localDate,localTime,watts,wattHours,tags
2014-03-01T08:02:30.561Z,11,Main,2014-03-01,21:02:30.561,820,44555,"foo,reset"
2014-03-01T08:03:30.545Z,11,Main,2014-03-01,21:03:30.545,835,44567,
2014-03-01T08:04:30.545Z,11,Main,2014-03-01,21:04:30.545,856,44578,

JSON output format

ID net.solarnetwork.central.datum.export.standard.JsonDatumExportOutputFormatService

The JSON output format encodes the export as a JSON array of datum JSON objects, with data properties as object keys. It does not support any settings. Example output looks like this:

[
  {"created":"2014-03-01 08:02:30.561Z","nodeId":11,"sourceId":"Main","localDate":"2014-03-01","localTime":"21:02","watts":820,"wattHours":44555,"tags":["foo","reset"]},
  {"created":"2014-03-01 08:03:30.545Z","nodeId":11,"sourceId":"Main","localDate":"2014-03-01","localTime":"21:03","watts":835,"wattHours":44567},
  {"created":"2014-03-01 08:04:30.545Z","nodeId":11,"sourceId":"Main","localDate":"2014-03-01","localTime":"21:04","watts":856,"wattHours":44578}
]

List output formats response

The response contains an array of output format service definitions.

Property Type Description
id string The service unique identifier.
settingSpecifiers array The configurable setting specifiers for the service.
locale string The locale of the localized messages in the response.
localizedName string A localized name for the service.
localizedDescription string A localized description of the service.
localizedInfoMessages object Localized messages for the associated setting specifiers.

An example response looks like:

{
  "success": true,
  "data": [
    {
      "id": "net.solarnetwork.central.datum.export.standard.CsvDatumExportOutputFormatService",
      "locale": "en-NZ",
      "settingSpecifiers": [
        {
          "key": "includeHeader",
          "defaultValue": true,
          "trueValue": true,
          "falseValue": false,
          "transient": false,
          "type": "net.solarnetwork.settings.ToggleSettingSpecifier"
        }
      ],
      "localizedDescription": "Export data in comma separated values (spreadsheet) format.",
      "localizedInfoMessages": {
        "includeHeader.key": "Include Header",
        "includeHeader.desc": "Toggle the inclusion of a CSV header row."
      },
      "localizedName": "CSV"
    },
    {
      "id": "net.solarnetwork.central.datum.export.standard.JsonDatumExportOutputFormatService",
      "locale": "en-NZ",
      "settingSpecifiers": [],
      "localizedDescription": "Export data in JSON format.",
      "localizedInfoMessages": {},
      "localizedName": "JSON"
    }
  ]
}
Add a custom footer
Clone this wiki locally