Autotune insulinPeakTime and insulinEndTime #1018

scottleibrand · 2018-05-22T05:12:30Z

This PR optionally (if oref0-autotune is run with --tune-insulin-curve=true ) allows autotune to find the best values for the insulinPeakTime and insulinEndTime parameters used by the rapid-acting and ultra-rapid exponential insulin activity curves. It does so by repeatedly re-running the autotune calculations for each insulinEndTime, +/- 2h from the current setting in 1h steps, and for each insulinPeakTime, +/- 10m from the current setting in 5m steps, and calculating the sum of the squared deviations categorized as basals. It then determines whether the run with the lowest sum of squared deviations was longer or shorter than the current setting, and if so, whether the next 1h/5m step in that direction is better than the current setting. If so, it tunes the setting by one 1h/5m step in the direction that showed improvement, and then repeats the process the next day.

Since pump settings do not correspond in any meaningful way to insulinPeakTime and insulinEndTime, and since very few users know how to tune those values manually, the safety bounds for autotuning insulinPeakTime are simply set as a range of acceptable values for each insulin activity curve type: 35-100m peak for ultra-rapid, and 50-120m for others. Both curves enforce a minimum insulinEndTime of 5h, with (currently) no maximum.

Terminology note: insulinEndTime is currently equated with DIA, but this may need to be changed to improve understanding. DIA, as used by Medtronic and others, refers to the duration over which 90% of insulin activity occurs. In the exponential insulin activity curve model, but the insulinEndTime parameter is the time after which insulin activity is modeled to reach zero. See LoopKit/Loop#388 (comment) for details on the exponential curves used in OpenAPS. As a result, the best-fit insulinEndTime is usually at least 2x as long as what users considered their DIA to be in normal pump use. Users should not be surprised if the insulinEndTime / DIA calculated by autotune is considerably longer than what they would use in their insulin pump. Such long times do not indicate a problem with the autotune process: they're simply a result of the fact that insulin activity exponentially decays and takes a really long time to get completely down to zero.

Fixed DIA to constant value for finding peak time.

Conflicts: bin/oref0-set-device-clocks.sh package.json

danamlewis · 2018-05-22T05:27:28Z

Discussion: what are reasonable peak time? I am not necessarily comfortable with the above proposed defaults, without testing this on a wider set of data to see the range of tuned values.

tim2000s · 2018-05-23T21:35:41Z

I think I need to build a dev rig that uses this and test it out. The ranges feel very wide. They may not be unreasonable, but given the data from clamp testing is generally a mean from a population, assuming a normal distribution I think these seem quite wide.

vanzaam · 2018-05-28T11:58:27Z

I've installed 0.7.0 dev(edison), but this code with autotune peaktime doesn't work. --tune-dia-and-peak=true - unknown option. but with --tune-insulin-curve=true
I have errors:

/root/src/oref0/lib/autotune-prep/index.js:30
let minDeviations = 1000000;
^^^^^^^^^^^^^
SyntaxError: Unexpected identifier
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/root/src/oref0/bin/oref0-autotune-prep.js:23:16)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
Could not run oref0-autotune-prep ns-treatments.2018-05-27.json profile.json ns-entries.2018-05-27.json

scottleibrand · 2018-05-28T12:03:29Z

If your nodejs doesn’t like let that means you didn’t use the most recent install script, which upgrades you to node 8. We could probably change all instances of let to var for backwards compatibility though.

ToreBj · 2018-05-31T10:49:16Z

Same symptoms as @vanzaam . Node -v gives v0.10.29. (which is 10?? or not?, version numbering confuses me)
One other thing is that when I a set the branch to tune-dia-and-peak and did git pull I was already "Already up-to-date."?

jpcunningh · 2018-05-31T14:40:42Z

The first number provides the major revision, so v0.10.29 is prior to version 1 which is very old, so you need to update to version 8 to fix the compatibility issue. Already-up-to-date from the git pull command means your local copy matches the copy in your GitHub branch, so you should be good to go.

…

On Thu, May 31, 2018 at 5:49 AM Tore Bjørndalen ***@***.***> wrote: Same symptoms as @vanzaam <https://github.com/vanzaam> . Node -v gives v0.10.29. (which is 10?? or not?, version numbering confuses me) One other thing is that when I a set the branch to tune-dia-and-peak and did git pull I was already "Already up-to-date."? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1018 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Ag8XaJjRq_b6SdKKCkDWH9pUcbnODBq1ks5t38qwgaJpZM4UH_K-> .

sulkaharo · 2018-05-31T15:58:28Z

When working on the new curves last summer, I noticed even an 10 minute change to the peak made a fairly large change in the maths, so going down to 35 minutes from he default peak of 55 sounds like a huge default delta. +/-20% tuning would be 10 minutes either way?

scottleibrand · 2018-05-31T16:45:40Z

+/-20% is a good rule of thumb for things that affect the total expected impact of insulin delivery, and therefore can directly result in +/-20% insulin delivery. The insulin timing parameters don't change the total area under the curve, though, so we can safely vary them much more, to better reflect the data, without risking hypos or DKA. We still need to run this algorithm (and various other fitness functions, such as RMS, sum of deviations, etc.) against multiple people's NS instances to see how much variance in insulinPeakTime and insulinEndTime we get from real-world data, but I expect that a 35 minute peak would be quite rare. It will probably be pretty common to see peaks extended by 20m or more, though.

ToreBj · 2018-05-31T17:44:19Z

@jpcunningh thanks. I installed v8. I ran with option --tune-insulin-curve=true not --tune-dia-and-peak=true as stated by @scottleibrand at the top.

DIA 5 total sum squared deviations: 673.094 (mg/dL)^2
DIA 6 total sum squared deviations: 817.908 (mg/dL)^2
DIA 7 total sum squared deviations: 1009.769 (mg/dL)^2
DIA 8 total sum squared deviations: 1020.913 (mg/dL)^2
Optimum DIA 4 total sum squared deviations: 673.094 (mg/dL)^2
insulinPeakTime 45 total sum squared deviations: 530.166 (mg/dL)^2
insulinPeakTime 50 total sum squared deviations: 704.835 (mg/dL)^2
insulinPeakTime 55 total sum squared deviations: 817.908 (mg/dL)^2
insulinPeakTime 60 total sum squared deviations: 995.634 (mg/dL)^2
insulinPeakTime 65 total sum squared deviations: 1012.991 (mg/dL)^2
Optimum insulinPeakTime 45 total sum squared deviations: 530.166 (mg/dL)^2
oref0-autotune-core autotune.2018-05-30.json profile.json profile.pump.json > newprofile.2018-05-30.json
Adjusting DIA from 6 to 5 hours
Adjusting insulinPeakTime from 55 to 50 minutes

Should I change DIA on the pump?

danamlewis · 2018-05-31T17:46:37Z

(Reminder - this branch is off of 0.7.0-dev. 0.7.0 and all related feature branches should not be used on primary rigs and should be closely monitored as they are highly WIP).

scottleibrand · 2018-05-31T17:49:00Z

I wouldn't make any changes to your pump at this point. We need to run this over multiple weeks worth of data, on multiple people's NS data, and see if it always converges on sensible values for insulinPeakTime and insulinEndTime.

I also fixed my issue summary to use --tune-insulin-curve=true

tim2000s · 2018-06-01T05:34:33Z

I’m looking forward to testing this. My gut feel is that over a longer period of time it should Autotune in some good mean peak and end times, but on shorter time frame data, we may find that timings change with ISF/exercise.

I haven’t read through the code fully yet, but is there a weighting factor applied, similarly to ISF and CR?

PieterGit · 2018-08-20T19:33:37Z

Just did a test run on a 0.7.0 rig.

Best insulinEndTime for meanDeviations: 1 hours
Best insulinEndTime for RMSDeviations: 1 hours
Leaving insulinEndTime unchanged at 3
Best insulinPeakTime for meanDeviations: 85 minutes
Best insulinPeakTime for RMSDeviations: 85 minutes
Adjusting insulinPeakTime from 75 to 80 minutes

Question: I would expect it to set insulinEndTime to 2 hours in this case.

scottleibrand · 2018-08-20T19:50:59Z

How did you get an insulinEndTime of 3? The minimum should be 5h.

It won't always move things in the direction of the "best" value: it just changes things one "step" in that direction if both the mean and RMSDeviations are better at the next step. In other words, if all four comparisons don't agree it's better to shorten (or lengthen) it, it will remain unchanged.

sethgagnon · 2018-08-20T20:03:50Z

I am using a standalone VM to run this, as I am open looping. Wanted to share to see if this helps testing at all, as I saw you mentioned var versus let earlier in the thread, I wasnt sure if this was related:

ref0-autotune-core autotune.2018-08-19.json profile.json profile.pump.json > newprofile.2018-08-19.json
Best insulinEndTime for meanDeviations: 6 hours
Best insulinEndTime for RMSDeviations: 6 hours
Adjusting insulinEndTime from 5 to 6 hours
/home/seth/src/oref0/lib/autotune/index.js:98
var currentPeakMeanDev = peakDeviations[2].meanDeviation;
^

TypeError: Cannot read property 'meanDeviation' of undefined
at tuneAllTheThings (/home/seth/src/oref0/lib/autotune/index.js:98:51)
at Object. (/home/seth/src/oref0/bin/oref0-autotune-core.js:60:27)
at Module._compile (module.js:410:26)
at Object.Module._extensions..js (module.js:417:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Function.Module.runMain (module.js:442:10)
at startup (node.js:136:18)
at node.js:966:3
false
Warning: API_SECRET is not set when calling oref0-autotune.sh
(this is only a problem if you have locked down read-only access to your NS).
Could not run oref0-autotune-core autotune.2018-08-19.json profile.json profile.pump.json

scottleibrand · 2018-08-20T20:07:08Z

@sethgagnon does autotune.2018-08-19.json look complete?

sethgagnon · 2018-08-20T20:20:30Z

@scottleibrand here are the files that may help:

autotune.2018-08-19.zip

scottleibrand · 2018-08-20T20:47:46Z

Thanks. I'll see if I can reproduce the problem locally with your NS URL and enable some debugging to see why you got that error.

sethgagnon · 2018-08-20T20:52:42Z

no prob. if its something just specific to me, dont worry about it. i know you have better things to be looking at then an issue that is only relevant to me.

…

On Mon, Aug 20, 2018 at 4:47 PM Scott Leibrand ***@***.***> wrote: Thanks. I'll see if I can reproduce the problem locally with your NS URL and enable some debugging to see why you got that error. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1018 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAkfxlwOkaNHj8XDkR7-_xM240qTowQxks5uSyB0gaJpZM4UH_K-> .

-- Seth Gagnon

scottleibrand · 2018-08-21T21:15:36Z

Try adding "insulinPeakTime": 75, to your profile.json and pumpprofile.json and see if that fixes it.

scottleibrand · 2018-08-21T21:20:58Z

I was able to reproduce the problem, and fix it with d6bf57d

tim2000s · 2018-08-22T05:41:18Z

Now that this is in the Dev branch, is there anywhere to document that for the DIA and Peak Time tuning, you need to set InsulinPeakTime in preferences.json to the correct value as it doesn’t pick this data up from the selected curve?

tim2000s · 2018-08-22T08:25:04Z

Just checked the output on my dev rig and it’s elected a peak of 75 on a Fiasp curve and with insulinPeakTime set to 55 and the log shows no reasons for the decision.

scottleibrand · 2018-08-22T18:37:56Z

@tim2000s can you paste the incorrect output, and the relevant settings files needed to reproduce?

tim2000s · 2018-08-22T21:52:37Z

preferences.json:

        "max_iob": 12,
        "max_daily_safety_multiplier": 5,
        "current_basal_safety_multiplier": 6,
        "autosens_max": 1.9,
        "autosens_min": 0.1,
        "rewind_resets_autosens": true,
        "autosens_adjust_targets": true,
        "adv_target_adjustments": true,
        "maxCOB": 180,
        "override_high_target_with_low": false,
        "skip_neutral_temps": true,
        "bolussnooze_dia_divisor": 7,
        "min_5m_carbimpact": 5,
        "carbratio_adjustmentratio": 1,
        "autotune_isf_adjustmentFraction": 0.5,
        "remainingCarbsCap": 90,
        "enableUAM": true,
        "enableSMB_with_bolus": true,
        "enableSMB_with_COB": true,
        "enableSMB_with_temptarget": true,
        "maxSMBBasalMinutes": 90,
        "curve": "ultra-rapid",
        "useCustomPeakTime": false,
        "insulinPeakTime": 55,
        "enableSMB_always": true, 
        "allowSMB_with_high_temptarget": true,
        "enableSMB_after_carbs": true,
        "high_temptarget_raises_sensitivity": true, 
        "low_temptarget_lowers_sensitivity": true,
        "sensitivity_raises_target": true,
        "resistance_lowers_target": true,
        "exercise_mode": true,
        "unsuspend_if_no_temp": true,
        "pushoverGlances": true,
        "MaxUAMSMBBasalMinutes": 60,
        "A52_risk_enable": true

and the log file from the run:

oldCR: 5.932 fullNewCR: 5.244 newCR: 5.588
p50deviation: 1.03 p50BGI -0.97 p50ratios: 0.195 Old ISF: 31.666 fullNewISF: 6.175 adjustedISF: 24.687 newISF: 30.27 newDIA: 7 newPeak: 75

Autotune pump profile recommendations:
---------------------------------------------------------
Recommendations Log File: /root/myopenaps//autotune/autotune_recommendations.log

/usr/local/bin/oref0-autotune-recommends-report: line 25: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.UTF-8): No such file or directory
Parameter      | Pump     | Autotune 
-------------------------------------
ISF [mg/dL/U]  | 43.200   | 30.270   
Carb Ratio[g/U]| 7.000    | 5.588    
Basals [U/hr]  | -        |
  00:00        | 0.850    | 0.740    
  01:00        | 0.800    | 0.725    
  02:00        | 0.750    | 0.681    
  03:00        | 0.650    | 0.645    
  04:00        | 0.700    | 0.665    
  05:00        | 0.850    | 0.750    
  06:00        | 0.900    | 0.700    
  07:00        | 0.900    | 0.718    
  08:00        | 0.900    | 0.702    
  09:00        | 0.900    | 0.758    
  10:00        | 0.900    | 0.762    
  11:00        | 0.900    | 0.765    
  12:00        | 0.900    | 0.763    
  13:00        | 0.950    | 0.764    
  14:00        | 0.900    | 0.764    
  15:00        | 0.950    | 0.770    
  16:00        | 0.900    | 0.758    
  17:00        | 0.900    | 0.756    
  18:00        |          | 0.729    
  19:00        | 0.950    | 0.732    
  20:00        | 0.850    | 0.754    
  21:00        | 0.850    | 0.817    
  22:00        | 0.850    | 0.831    
  23:00        | 0.900    | 0.816

There is no data in the file for the DIA or peak adjustments.

scottleibrand · 2018-08-22T22:40:37Z

Just pushed a potential fix to dev (thanks @PieterGit for pointing out the missing === true stuff): @tim2000s can you re-test? If it's still missing insulinPeakTime, can you also include the output of grep -h insulin autotune/autotune.2018-08-2*.log | tail -20

sethgagnon · 2018-08-23T15:19:20Z

If we are open looping using AndroidAPS, how can we edit our peak time to have the oref algo use it?

scottleibrand · 2018-08-23T15:56:09Z

I’m not familiar with that detail of the AndroidAPS implementation, so you might want to ask in their Gitter.

TwistaTim · 2018-08-24T02:12:56Z

@sethgagnon: yes its called free peak oref, you set the peaktime in the settings....

tim2000s · 2018-08-24T08:07:18Z

@scottleibrand - just realised that the lack of info about insulin action times was user error. Rectified that and will report the outcome. Had assumed that action time calculation was set to automatically true as they were being output in the log, but they still need flagging.

Does raise a concern that they default to DIA 7 hours and peak at 75 mins even when not being run though, rather than picking up defaults from the preferences.json file.

tim2000s · 2018-08-24T13:10:36Z

Okay, the re-run has caused a change in DIA and peak, once I'd corrected the settings, but I'm still seeing it starting at 75mins on Fiasp. It looks as though PieterGit’s change is now stopping the expression from using the custom insulin peak time, but hasn’t helped it to pick up the correct curve from preferences. .

p50deviation: 1.51 p50BGI -3.61 p50ratios: 0.606 Old ISF: 35.122 fullNewISF: 21.284 adjustedISF: 32.242 n
ewISF: 34.546 newDIA: 6 newPeak: 70

Autotune pump profile recommendations:
---------------------------------------------------------
Recommendations Log File: /root/myopenaps//autotune/autotune_recommendations.log

and output from the grep:

insulinEndTime 5 meanDeviation: 1.337 SMRDeviation: 0.752 RMSDeviation: 2.011 (mg/dL)
insulinEndTime 6 meanDeviation: 1.272 SMRDeviation: 0.689 RMSDeviation: 1.964 (mg/dL)
insulinEndTime 7 meanDeviation: 1.296 SMRDeviation: 0.692 RMSDeviation: 2.013 (mg/dL)
insulinEndTime 8 meanDeviation: 1.297 SMRDeviation: 0.692 RMSDeviation: 2.029 (mg/dL)
insulinEndTime 9 meanDeviation: 1.296 SMRDeviation: 0.684 RMSDeviation: 2.043 (mg/dL)
insulinPeakTime 65 meanDeviation: 1.236 SMRDeviation: 0.68 RMSDeviation: 1.886 (mg/dL)
insulinPeakTime 70 meanDeviation: 1.253 SMRDeviation: 0.67 RMSDeviation: 1.941 (mg/dL)
insulinPeakTime 75 meanDeviation: 1.296 SMRDeviation: 0.692 RMSDeviation: 2.013 (mg/dL)
insulinPeakTime 80 meanDeviation: 1.319 SMRDeviation: 0.705 RMSDeviation: 2.063 (mg/dL)
insulinPeakTime 85 meanDeviation: 1.342 SMRDeviation: 0.71 RMSDeviation: 2.116 (mg/dL)
Best insulinEndTime for meanDeviations: 6 hours
Best insulinEndTime for RMSDeviations: 6 hours
Adjusting insulinEndTime from 7 to 6 hours
Best insulinPeakTime for meanDeviations: 65 minutes
Best insulinPeakTime for RMSDeviations: 65 minutes
Adjusting insulinPeakTime from 75 to 70 minutes

tim2000s · 2018-08-24T15:28:33Z

One thing I’ve run into with this is that my rig overheated (no airflow issues) while charging and calculating insulinPeakTime sonit stopped looping.

tim2000s · 2018-08-24T15:53:36Z

I’ve spent a fair bit of time playing with this this afternoon. The logic for invoking the 55 minute peak time seems to be evaluating to false constantly, and I think this is because it is not evaluating the opts.profile.curve === "ultra-rapid" expression correctly, as it defaults to 75, which suggests that it is evaluating the customPeakTime expression without issue.

Not sure why not though, as I’ve tried multiple iterations of it this afternoon.

scottleibrand · 2018-08-24T15:55:25Z

Might want to double check your input files: grep -r curve * | grep json | grep rapid

tim2000s · 2018-08-24T23:35:20Z

This was the output:

preferences.json: "curve": "ultra-rapid",
settings/pumpprofile.json: "curve": "ultra-rapid",
settings/autotune.json: "curve": "rapid-acting",
settings/profile.json: "curve": "ultra-rapid”,

I think this is a similar issue to the one where you switch from >1 ISF, in that Autotune reuses its last version of a profile, so even when you update preferences, it perpetuates incorrect data.

scottleibrand · 2018-08-24T23:53:43Z

Yep, looks like it. Do we have an issue open on that? Want to work with me on figuring out and fixing the root cause?

tim2000s · 2018-08-25T12:22:24Z

I’ll get one raised and happy to.

PieterGit · 2018-12-20T06:45:44Z

@tim2000s @scottleibrand Just reading these issues again. I think the issue you raised with the stale autotune profile at #1018 (comment) is managed with issue #1093 and resolved with @jpcunningh PR #1101

@tim2000s Can you confirm that this is correct?

tim2000s · 2018-12-20T21:56:56Z

I believe so. I never double checked though. @jpcunningh should be able to confirm.

jpcunningh · 2018-12-21T01:34:07Z

@tim2000s, @PieterGit, and @scottleibrand, #1101 doesn't maintain the curve, but something very similar could be submitted that pulls the curve from the pumpprofile.

I created PR #1174 to do that. Comments are welcome as I haven't spent a lot of time thinking through all of the scenarios.

Jeremy Cunningham and others added 13 commits May 20, 2018 20:53

Initial commit of tuning dia and peak capability.

3a4a81c

Added setting customPeakTime to true to allow peak time manipulation.

6fd9549

Fixed Math function being called.

6817bc7

Only allowed DIA to range to 9.

a55277d

Fixed DIA to constant value for finding peak time.

skip glucose records w/o valid glucose

c1f0653

Merge branch 'autotune' into tune-dia-and-peak

f64da70

Conflicts: bin/oref0-set-device-clocks.sh package.json

consider DIAs +/- 2h, and peaks +/- 10m, each run

620cb0c

use correct peaks; store output

35e9b14

tune DIA and insulinPeakTime

a53ec46

comment out debugging

8e720b1

only tune DIA and peak if requested

35a1edd

allow 35-100m peak for fiasp, 50-120m for others

0771b92

syntax

0987469

scottleibrand changed the title ~~Autotune DIA and peak~~ Autotune insulinPeakTime and insulinEndTime May 22, 2018

scottleibrand added 4 commits June 3, 2018 16:26

replace let with var for compatibility

7e73528

round deviations before comparison

8b37073

require 1% improvement to change DIA/peak

14a189c

fix variable reference

cf35f41

scottleibrand deleted the tune-dia-and-peak branch August 21, 2018 00:41

jpcunningh mentioned this pull request Dec 21, 2018

Always keep insulin curve value up to date in autotune profile. #1174

Merged

Autotune insulinPeakTime and insulinEndTime #1018

Autotune insulinPeakTime and insulinEndTime #1018

Conversation

scottleibrand commented May 22, 2018 • edited Loading

danamlewis commented May 22, 2018

tim2000s commented May 23, 2018

vanzaam commented May 28, 2018

scottleibrand commented May 28, 2018

ToreBj commented May 31, 2018

jpcunningh commented May 31, 2018 via email

sulkaharo commented May 31, 2018

scottleibrand commented May 31, 2018

ToreBj commented May 31, 2018 • edited Loading

danamlewis commented May 31, 2018

scottleibrand commented May 31, 2018 • edited Loading

tim2000s commented Jun 1, 2018

PieterGit commented Aug 20, 2018

scottleibrand commented Aug 20, 2018

sethgagnon commented Aug 20, 2018 • edited Loading

scottleibrand commented Aug 20, 2018

sethgagnon commented Aug 20, 2018 • edited Loading

scottleibrand commented Aug 20, 2018

sethgagnon commented Aug 20, 2018 via email

scottleibrand commented Aug 21, 2018 • edited Loading

scottleibrand commented Aug 21, 2018

tim2000s commented Aug 22, 2018

tim2000s commented Aug 22, 2018 • edited Loading

scottleibrand commented Aug 22, 2018

tim2000s commented Aug 22, 2018

scottleibrand commented Aug 22, 2018

sethgagnon commented Aug 23, 2018

scottleibrand commented Aug 23, 2018

TwistaTim commented Aug 24, 2018

tim2000s commented Aug 24, 2018

tim2000s commented Aug 24, 2018 • edited Loading

tim2000s commented Aug 24, 2018

tim2000s commented Aug 24, 2018 • edited Loading

scottleibrand commented Aug 24, 2018

tim2000s commented Aug 24, 2018

scottleibrand commented Aug 24, 2018

tim2000s commented Aug 25, 2018

PieterGit commented Dec 20, 2018 • edited Loading

tim2000s commented Dec 20, 2018

jpcunningh commented Dec 21, 2018

scottleibrand commented May 22, 2018 •

edited

Loading

ToreBj commented May 31, 2018 •

edited

Loading

scottleibrand commented May 31, 2018 •

edited

Loading

sethgagnon commented Aug 20, 2018 •

edited

Loading

sethgagnon commented Aug 20, 2018 •

edited

Loading

scottleibrand commented Aug 21, 2018 •

edited

Loading

tim2000s commented Aug 22, 2018 •

edited

Loading

tim2000s commented Aug 24, 2018 •

edited

Loading

tim2000s commented Aug 24, 2018 •

edited

Loading

PieterGit commented Dec 20, 2018 •

edited

Loading