Command Line Tools

Description of talking to Paco server using Curl

Introduction

You can both retrieve and post events to a PacoServer via http. You can also retrieve experiments from a PacoServer via http.

Details

These examples use http://curl.haxx.se/.

First you need to establish an oauth cookie to Appengine via Google's OAuth2 interface.

Here is a script that can establish an OAuth2 cookie.

To use it you must first go to the Google developer console and generate a cliendId and secret. Follow the instructions at https://developers.google.com/identity/protocols/OAuth2InstalledApp

set_oauth.sh https://github.com/google/paco/blob/develop/Paco-Server/scripts/set_oauth.sh

#!/bin/sh

# using instructions from
# https://developers.google.com/identity/protocols/OAuth2InstalledApp

# #1 generate credential -- see above page for steps to generate client id & secret
# hint: use the installed app option (Other under API/Credentials in Google dev console)

echo "enter client id from Google developer console"
read client_id

open "https://accounts.google.com/o/oauth2/auth?response_type=code&scope=email&redirect_uri=urn:ietf:wg:oauth:2.0:oob&client_id=$client_id"

echo "enter authcode from browser authentication page"
read auth_code

echo "enter client secret from Google Developer console"
read client_secret

# instead of the all-in-one-line approach here, we might want to capture the response then slice and dice in case there was an error
export refresh_token=`curl https://www.googleapis.com/oauth2/v3/token \
     -d code=$auth_code \
     -d client_id=$client_id \
     -d client_secret=$client_secret \
     -d redirect_uri=urn:ietf:wg:oauth:2.0:oob \
     -d grant_type=authorization_code | grep 'refresh_token' | cut -d ":" -f 2 | cut -c 2- | rev | cut -c 2- | rev`

echo "refresh token: $refresh_token"

# save the refresh token somewhere to prevent having to redo this step for each request.
# then individual request files can read the refresh token and request access_tokens (which expire)
echo $refresh_token > ~/.qs_paco
chmod 600 ~/.qs_paco

echo "tried to save refresh token in ~/.qs_paco. Now you should be able to run one of the Paco cmdline scripts"

Make the script executable, chmod 700, then run it,

    %> ./set_oauth.sh
    
=> enter client id from Google developer console
%> a1b2c3d4.apps.googleusercontent.com

// this will open the browser and ask you to grant the script access to the pacoapp.
// if you accept, it will give you an authcode. Paste that in the line below

=> enter authcode from browser authentication page
%> 123456abcdefg

=> enter client secret from Google Developer console
%> client_secret=abcdefgh

That should set up a private file with your auth credentials. Now you can retrieve event data from Paco -- unless your token expires, in which case, re-run this script.

Retrieving Events https://github.com/google/paco/blob/develop/Paco-Server/scripts/getpaco_qs.sh

getpaco_qs.sh This script expects that you have setup environment variables for the client_id and the client_secret. You could also just paste them in directly instead.

If you want to set them on the commandline do the following

%> export client_id=a1b2c3.apps.googleusercontent.com
%> export client_secret=abcdefghh12354

#/bin/bash

# be sure to generate a stored refresh token by running set_oauth.sh first (should only need it once unless you revoke the token)

# Read the stored refresh token. Get a new access token with it. Then, finally, doing something with a Paco endpoint.
refresh_token=`cat ~/.qs_paco`

export refresh_token=`sed -e 's/^"//' -e 's/"$//' <<< $refresh_token`

# get a new access token from the refresh token
export new_access_token_response=`curl https://www.googleapis.com/oauth2/v3/token \
     -d client_id=$client_id \
     -d client_secret=$client_secret \
     -d refresh_token=$refresh_token \
     -d grant_type=refresh_token`

#echo "nat: $new_access_token_response"
export access_token=`echo $new_access_token_response | grep 'access_token' | cut -d ":" -f 2 | cut -d "," -f 1 | tr -d '" '`
#echo "parsed at: ${access_token}"


#echo "new_access_token: $access_token"
# now you can access data with this token

echo "enter experimentId (from definition page on server)"
read experiment_id

echo "enter report format (csv,json,html)"
read report_format
 
echo "output file name"
read outfile

if [ "$report_format" = "json" ]; then
    result=`curl  -H "Authorization: Bearer $access_token" -L "https://quantifiedself.appspot.com/events?q=experimentId=$experiment_id&json"`
else
    joburl=`curl  -H "Authorization: Bearer $access_token" -L -v "https://quantifiedself.appspot.com/events?q=experimentId=$experiment_id&$report_format" 2>&1 | grep "GET /jobStatus" | cut -d " " -f 3`

    joburl="$joburl&cmdline=1"

    result=`curl  -H "Authorization: Bearer $access_token" -L "https://quantifiedself.appspot.com$joburl"`

    # refresh until the report is ready
    while [ "$result" = pending ]; do
        result=`curl  -H "Authorization: Bearer $access_token" -L "https://quantifiedself.appspot.com$joburl"`
    done
fi

echo "$result" > $outfile
    

Run this command and follow the prompts.

%> ./getpaco.sh 

=> enter experimentId (from definition page on server)
%> 4585324601671680

=> enter report format (csv,json,html)
%>csv

=> output file name
%> output.csv