Self update #1097

crossan007 · 2016-10-05T22:23:11Z

Should help with bugs like:
#1059
#875

Might Help with:
#524

Probably not quite ready for prime time, but I'd like to have it in 2.3.0

This adds capability for ChurchCRM to update itself from the latest github release.

It also adds signature verification for all source files in the project.

during Phing build, generateSignatures.php create a "signatures.json" file with a sha1 of every php and js file in the project.

at runtime, the system service will periodically call verifyApplicationIntegrity() to ensure that the files installed match the signature definition file that was generated at build.

If there are errors, then integrityCheck.json will cause the TaskService to display an alert that there are possibly corrupt files.

DawoudIO

looks good, watch your install dir and your gettext

DawoudIO · 2016-10-07T04:14:02Z

src/Service/SystemService.php

+    $UpgradeDir = $CRMInstallRoot."/Upgrade";
+    $url = $release['assets'][0]['browser_download_url'];
+    mkdir($UpgradeDir);
+    //file_put_contents($UpgradeDir."/".basename($url), file_get_contents($url));


why commented out

DawoudIO · 2016-10-07T04:14:37Z

src/Service/SystemService.php

+        for($i = 0; $i < $zip->numFiles; $i++) 
+        {
+            $archivedFileName = $zip->getNameIndex($i);
+            $targetFilename = str_replace("churchcrm/","/",$archivedFileName);


what are you trying to do here

we don't want to maintain the "churchcrm" folder inside of the zip archive when we extract it.

DawoudIO · 2016-10-07T04:15:50Z

src/Service/TaskService.php

+    }
+
+    if($integrityCheckData->status == "failure") {
+      array_push($tasks, $this->addTask("Application Integrity Check Failed", "/integrityCheck.json", true));


where are you sending the user

right now I'm showing them an ugly json file with details about why the integrity check failed.

This is the UI i still have to build

DawoudIO · 2016-10-07T04:16:01Z

src/UpgradeCRM.php

+require 'Include/Functions.php';
+
+// Set the page title and include HTML header
+$sPageTitle = "Upgrade ChurchCRM";


DawoudIO · 2016-10-07T04:16:15Z

src/UpgradeCRM.php

+
+<div class="login-box">
+    <div class="login-logo">
+        Upgrade <b>Church</b>CRM</a>


DawoudIO · 2016-10-07T04:16:30Z

src/UpgradeCRM.php

+    </div>
+    <div class="login-box-body" style="display:none" id="finalPhase">
+      <p class="login-box-msg"><?= gettext('Step 4: Login') ?></p>
+      <a href="Login.php?Logoff=True" class="btn btn-primary">Login to Upgraded System</a>


DawoudIO · 2016-10-07T04:16:37Z

src/UpgradeCRM.php

+      console.log(data);
+      var downloadButton = "<button class=\"btn btn-primary\" id=\"downloadbutton\" role=\"button\" onclick=\"javascript:downloadbutton('"+data.filename+"')\"><i class='fa fa-download'></i>  "+data.filename+"</button>";
+      $("#backupstatus").css("color","green");
+      $("#backupstatus").html("Backup Complete, Ready for Download.");


DawoudIO · 2016-10-07T04:16:57Z

src/UpgradeCRM.php

+ $("#applyUpdate").click(function(){
+   $.ajax({
+      type : 'POST',
+      url  : window.CRM.root +'/api/systemupgrade/doUpgrade', // the url where we want to POST


not sure rest API has case

DawoudIO · 2016-10-07T04:17:22Z

src/api/routes/systemupgrade.php

+
+$app->group('/systemupgrade', function () {
+
+  $this->get('/downloadLatestRelease', function () {


all lower case or -

DawoudIO · 2016-10-07T04:17:29Z

src/api/routes/systemupgrade.php

+    echo json_encode($upgradeFile);
+  });
+
+  $this->post('/doUpgrade', function ($request, $response, $args) {


…elf-update

DawoudIO · 2016-10-08T19:44:36Z

how do you test this ;)

crossan007 · 2016-10-08T21:34:22Z

That's a very good question.

I think there's a small glitch in the getLatestRelese() function that causes the app to "upgrade" if the latest version hosted on GitHub is less than the current build (i.e. our dev environments will always ask us to upgrade to whatever the last published version was).

Using this bug, you can upgrade your dev environment (2.3.0) to the latest release (2.2.3).

You should be able to just follow the prompts.

After it's complete, you should have a completely broken system, since downgrading the app is not supported :-)

DawoudIO · 2016-10-15T02:43:08Z

src/IntegrityCheck.php

+
+//Set the page title
+$sPageTitle = gettext('Integrity Check Results');
+if (!$_SESSION['bFinance'])


I don't get this check

Whooopss.. That's what i get for copying / pasting a page. I should get rid of the header too...

DawoudIO · 2016-10-15T02:43:49Z

src/IntegrityCheck.php

+  exit;
+}
+require 'Include/Header.php'; 
+$CRMInstallRoot = __DIR__;


no need for the var

DawoudIO · 2016-10-15T02:44:20Z

src/IntegrityCheck.php

+?>
+
+<div class="box box-body">
+  <p><?= gettext("Previous Integrity Check Result:") ?>


use the callout warning css to get read or green

fixed in a commit coming soon to a Repo near you

DawoudIO · 2016-10-15T02:45:05Z

src/Service/SystemService.php

+      $diff = $previous->diff($now);  // calculate the difference.
+      if (!$sLastIntegrityCheckTimeStamp || $diff->h >= $sIntegrityCheckInterval)  // if there was no previous backup, or if the interval suggests we do a backup now.
+      {
+        $CRMInstallRoot = dirname(__DIR__);


temp var not used else where

DawoudIO · 2016-10-15T02:47:23Z

src/mysql/install/Install.sql

+(1043, 'sQBDTSettings', @JSONV , 'json', @JSONV , 'QuickBooks Deposit Ticket Settings', 'ChurchInfoReport', 'Step7', NULL),
+(1044, 'sEnableIntegrityCheck', '1', 'boolean', '1', 'Enable Integrity Check', 'General', "Step5", NULL),
+(1045, 'sIntegrityCheckInterval', '1', 'Text', '1', 'Interval in Hours for Integrity Check', 'General', "Step5", NULL),
+(1046, 'sLastIntegrityCheckTimeStamp', '', 'Text', '', 'Last Integrity Check Timestamp', 'General', "Step5", NULL);


why not timestamp type?

We don't have a "timestamp" type available as a type in this context.

These are not SQL column types, but rather types enumerated in lines 185-221 of SystemSettings.php

we can track this separately, as there are a few other places where we may want to use a TimeStamp

Tracking this in #1140

DawoudIO · 2016-10-15T02:47:58Z

src/mysql/install/Install.sql

-(1043, 'sQBDTSettings', @JSONV , 'json', @JSONV , 'QuickBooks Deposit Ticket Settings', 'ChurchInfoReport', 'Step7', NULL);
+(1043, 'sQBDTSettings', @JSONV , 'json', @JSONV , 'QuickBooks Deposit Ticket Settings', 'ChurchInfoReport', 'Step7', NULL),
+(1044, 'sEnableIntegrityCheck', '1', 'boolean', '1', 'Enable Integrity Check', 'General', "Step5", NULL),
+(1045, 'sIntegrityCheckInterval', '1', 'Text', '1', 'Interval in Hours for Integrity Check', 'General', "Step5", NULL),


1hr is too fast... 1 day or 1 week... i lean to 1 week... maybe a drop down of options

for MVP, let's stick with "hours" as the unit. We can make it like 24, 48, or 168

DawoudIO · 2016-10-15T04:05:03Z

Clicking "Fetch Update Files" needs a spinner or something... i was not sure what was going on...
Same for "Apply Update Files"

DawoudIO · 2016-10-15T04:05:36Z

src/UpgradeCRM.php

+        <li><? gettext("Full Path:")?> <span id="updateFullPath"> </span></li>
+        <li><? gettext("SHA1:")?> <span id="updateSHA1"> </span></li>
+      </ul>
+      <input type="button" class="btn btn-warning" id="applyUpdate" <?= 'value="' . gettext("Apply Update Files") . '"' ?>>


This should be called.... upgrade system

DawoudIO · 2016-10-15T04:06:21Z

src/UpgradeCRM.php

+        <li><? gettext("Full Path:")?> <span id="updateFullPath"> </span></li>
+        <li><? gettext("SHA1:")?> <span id="updateSHA1"> </span></li>
+      </ul>
+      <input type="button" class="btn btn-warning" id="applyUpdate" <?= 'value="' . gettext("Apply Update Files") . '"' ?>>


why is "value" in php echo tag

DawoudIO · 2016-10-15T04:10:19Z

src/api/routes/systemupgrade.php

+  $this->post('/doupgrade', function ($request, $response, $args) {
+    $input = (object)$request->getParsedBody();
+    $upgradeResult = $this->SystemService->doUpgrade($input->fullPath,$input->sha1);
+    echo json_encode($upgradeResult);


where should we delete the zip file.

it happens in SystemService doUpgrade()

DawoudIO · 2016-10-15T04:10:51Z

address the issues above and we should be able to merge this soon

do integrity check from setup page fix setup page theme misc fixes.

crossan007 added 2 commits October 4, 2016 21:42

Self upgrade proof of concept

bfebff8

pre-test commit, since you know...this will overwrite my work.

0aee63c

crossan007 added the do not merge! label Oct 5, 2016

crossan007 added this to the 2.3.0 milestone Oct 5, 2016

crossan007 self-assigned this Oct 5, 2016

crossan007 added 4 commits October 6, 2016 22:34

Add integrity check

07257c1

api cleanup

a0399fa

update build script to generate signature file

a65324f

fix line ending

720ce83

DawoudIO reviewed Oct 7, 2016

View reviewed changes

DawoudIO added enhancement build labels Oct 7, 2016

crossan007 added 8 commits October 7, 2016 17:52

tweaks

f542a89

require admin rights to upgrade

07fb247

Merge branch 'develop' into self-update

1c1f5e6

fix derp

a2062a7

Merge branch 'self-update' of https://github.com/ChurchCRM/CRM into s…

a00572c

…elf-update

Merge branch 'develop' into self-update

28f2449

tweaks

0f3a4cf

Merge origin/self-update into self-update

919c6fa

DawoudIO added in progress and removed in progress labels Oct 8, 2016

crossan007 added 3 commits October 8, 2016 10:03

Merge branch 'develop' into self-update

4e514ea

improve integrity check

21a9369

Merge origin/self-update into self-update

b299280

crossan007 removed the do not merge! label Oct 8, 2016

crossan007 mentioned this pull request Oct 12, 2016

Detect missing dependencies #1123

Closed

crossan007 and others added 6 commits October 11, 2016 22:47

underp link for integrity check

0251942

Merge branch 'develop' into self-update

b913c26

show more detail in integritycheck.php

686d3a7

Merge origin/self-update into self-update

529b1e1

cleanup build xml

0f24dd8

Merge branch 'develop' into self-update

31165a3

DawoudIO reviewed Oct 15, 2016

View reviewed changes

DawoudIO self-assigned this Oct 15, 2016

crossan007 added 8 commits October 15, 2016 11:12

cleanup integrity check notification

ce312db

do integrity check from setup page fix setup page theme misc fixes.

delete zip

7ea7df0

Improve update efficiency. Runs much faster

24da903

fix issue with deep dirs

c56879f

weekly integrity check instead of hourly

4f0a63d

UI Improvements

fd582fd

UI Improvements

4830907

Handle missing files and fix sRootPath

41d99c8

crossan007 merged commit 84969cd into develop Oct 16, 2016

crossan007 deleted the self-update branch October 16, 2016 03:58

crossan007 removed the In Review label Oct 16, 2016


		$app->group('/systemupgrade', function () {

		$this->get('/downloadLatestRelease', function () {

Self update #1097

Self update #1097

Conversation

crossan007 commented Oct 5, 2016 • edited Loading

DawoudIO left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DawoudIO commented Oct 8, 2016

crossan007 commented Oct 8, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DawoudIO commented Oct 15, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DawoudIO commented Oct 15, 2016

crossan007 commented Oct 5, 2016 •

edited

Loading