[WIP] Upgrade material UI

docs/HOWTO_TEXTING_HOURS_ENFORCEMENT.md

@@ -28,7 +28,7 @@ The timezone in New York City is specified by the string `US/Eastern`. Other tim
 
 If the `TZ` environment variable is not set, Spoke will infer a time zone for each contact from the contact's ZIP code.
 
-If a contact does not have a ZIP code, Spoke will assume the contact is in New York City and use the local time there to determine if it's OK to send texts.
+If a contact does not have a ZIP code, Spoke will default to a 'narrow' range for the continental US, 12pm EST/9am PST - 9pm EST/6pm PST, and use that time to determine if it's OK to send texts.
 
 ## Is it Daylight Savings Time?
 

docs/HOWTO_setup_twilio_amazon_SQS.md

@@ -0,0 +1,73 @@
+
+# Handling high volume inbound SMS & Webhooks with Twilio Functions & Amazon SQS
+
+### Instead of making this the primary means of receiving messages, you can set the Twilio messaging service fallback to the SQS queue. 
+If your Spoke application is down/not-responding, then Spoke doesn't lose messages and they are added to the SQS queue. 
+In this scenario, we do NOT automate it, but when service is restored, we can manually run the trigger and load in the sqs messages from Twilio without having lost any incoming messages.
+* Due to an existing bug, we DO NOT recommend scheduling the following job yet:
+- recurring Lambda (technically CloudWatch) trigger enabled to reguarly look for incoming messages:
+https://github.com/MoveOnOrg/Spoke/blob/main/docs/HOWTO_DEPLOYING_AWS_LAMBDA.md#setting-up-scheduled-jobs
+
+### Instructions Below for Set Up
+* Follow the instructions in this link [Instructions for Set Up on Twilio.com] (https://www.twilio.com/blog/2017/07/handling-high-volume-inbound-sms-and-webhooks-with-twilio-functions-and-amazon-sqs.html)
+* TWILIO_SQS_QUEUE_URL should be the 'https://sqs.CHANGEME.amazonaws.com/CHANGEME` url (not the twilio function)
+* Set up AWS creds in the app
+
+
+Twilio function source here (must change the line `const INCOMING_SMS_URL = 'https://sqs.CHANGEME.amazonaws.com/CHANGEME` and maybe `region: 'us-west-1' `):
+```
+/* global exports, require, console, process, Twilio */
+'use strict'
+
+// Some Node.js modules are preinstalled in the system environment.
+// As of this writing, the third party modules are not configureable, but
+// they should be soon. For now, though, you can take advantage of
+// the AWS SDK being preinstalled. Require and initialize it here with the
+// IAM credentials in your system environment.
+const AWS = require('aws-sdk')
+AWS.config.update({
+  accessKeyId: process.env.AWS_KEY,
+  secretAccessKey: process.env.AWS_SECRET
+})
+
+// Get a handle to SQS in the AWS region you created it in
+let sqs = new AWS.SQS({ region: 'us-west-1' })
+
+// Define SQS queue URLs from your AWS account
+const INCOMING_SMS_URL = 'https://sqs.CHANGEME.amazonaws.com/CHANGEME
+// const STATUS_CALLBACK_URL = 'https://sqs.us-east-2.amazonaws.com/XXX/status-callbacks'
+
+// Implement handler function for incoming messages and status callbacks
+exports.handler = function(context, event, callback) {
+  // SQS send params - assume it's a status callback to a standard queue
+  let sendParameters = {
+    MessageBody: JSON.stringify(event)
+    //QueueUrl: STATUS_CALLBACK_URL
+  }
+
+  // If the MessageStatus parameter is not passed, this is an incoming SMS
+  // message - add appropriate SQS parameters and change the URL
+  if (!event.MessageStatus) {
+      sendParameters.QueueUrl = INCOMING_SMS_URL
+      // FIFO queues use the message group ID to return messages to consumers
+      // in logical groups. For an SMS app, a good group ID is the recipient
+      // phone number
+      // sendParameters.MessageGroupId = event.From.replace(/\D/g,'')
+
+
+    // Add the message to the appropriate SQS queue
+    sqs.sendMessage(sendParameters, (err, res) => {
+      // For now, we'll just log any errors SQS throws us
+      console.log(err)
+      console.log(res)
+
+      // Send a TwiML response with no reply message - we'll handle 
+      // any responses from our workers
+      let twiml = new Twilio.twiml.MessagingResponse()
+      callback(null, twiml)
+    })
+  }
+}
+```
+
+

docs/REFERENCE-best-practices-conformance-messaging.md

@@ -70,6 +70,6 @@ While texters should be able to send messages they desire, administrators can ch
 ## Blocking inappropriate texter content
 Some language should not be possible to be texted out. This includes racial and misogynistic slurs, for example. Not on by default, but we recommend to consider enabling MESSAGE_HANDLERS=profanity-tagger along with PROFANITY_TEXTER_BLOCK_SEND=1:
 
-* PROFANITY_TEXTER_REGEX_BASE64 controls which phrases are blocked with a regular expression.  You can see the default (encoded) in the [Spoke profanity-tagger code base](https://github.com/MoveOnOrg/Spoke/blob/main/src/integrations/message-handlers/profanity-tagger/index.js#L4). Take care to thoroughly test your regular expression before deploying a change.
+* PROFANITY_TEXTER_REGEX_BASE64 controls which phrases are blocked with a regular expression.  You can see the default (encoded) in the [Spoke profanity-tagger code base](https://github.com/MoveOnOrg/Spoke/blob/main/src/extensions/message-handlers/profanity-tagger/index.js). Take care to thoroughly test your regular expression before deploying a change.
 
 

docs/_sidebar.md

@@ -49,7 +49,3 @@
   - [How access roles work in Spoke](REFERENCE_ROLES_DESCRIPTION.md)
   - [How to scale Spoke](HOWTO-scale-spoke-plan.md)
   - [How to set up Spoke with Postgresql](HOWTO_USE_POSTGRESQL.md)
-
-- Outlines of docs we wish we had - contributors welcome!
-
-  - [MESSAGE REVIEW documentation](EXPLANATION_MESSAGE_REVIEW.md)

src/components/CampaignContactsChoiceForm.jsx

@@ -82,7 +82,8 @@ export class CampaignContactsChoiceForm extends React.Component {
       maxNumbersPerCampaign,
       contactsPerPhoneNumber,
       ingestMethodChoices,
-      pastIngestMethod
+      pastIngestMethod,
+      ensureComplete
     } = this.props;
     const ingestMethod = this.getCurrentMethod();
     const ingestMethodName = ingestMethod && ingestMethod.name;
@@ -91,6 +92,49 @@ export class CampaignContactsChoiceForm extends React.Component {
         ? pastIngestMethod
         : null;
     const IngestComponent = components[ingestMethodName];
+    if (ensureComplete) {
+      // isStarted
+      return (
+        <div>
+          {this.props.contactsCount && (
+            <div>
+              <div>Ingest Method: {ingestMethod.displayName}</div>
+              <div>Loaded Contacts: {this.props.contactsCount}</div>
+              {lastResult ? (
+                <div>
+                  <div>Deleted Duplicates: {lastResult.deletedDupes}</div>
+                  <div>Deleted OptOuts: {lastResult.deletedOptouts}</div>
+                </div>
+              ) : null}
+            </div>
+          )}
+          {this.props.jobResultMessage && (
+            <div>
+              <CampaignFormSectionHeading title="Job Outcome" />
+              <div>{this.props.jobResultMessage}</div>
+            </div>
+          )}
+
+          {IngestComponent && IngestComponent.prototype.renderAfterStart ? (
+            <IngestComponent
+              onChange={chg => {
+                this.handleChange(chg);
+              }}
+              onSubmit={this.props.onSubmit}
+              campaignIsStarted={ensureComplete}
+              icons={icons}
+              saveDisabled={this.props.saveDisabled}
+              saveLabel={this.props.saveLabel}
+              clientChoiceData={ingestMethod && ingestMethod.clientChoiceData}
+              lastResult={lastResult}
+              jobResultMessage={null}
+              contactsPerPhoneNumber={contactsPerPhoneNumber}
+              maxNumbersPerCampaign={maxNumbersPerCampaign}
+            />
+          ) : null}
+        </div>
+      );
+    }
 
     return (
       <div>
@@ -169,7 +213,7 @@ export class CampaignContactsChoiceForm extends React.Component {
                 this.handleChange(chg);
               }}
               onSubmit={this.props.onSubmit}
-              campaignIsStarted={this.props.ensureComplete}
+              campaignIsStarted={ensureComplete}
               icons={icons}
               saveDisabled={this.props.saveDisabled}
               saveLabel={this.props.saveLabel}

src/containers/AdminCampaignEdit.jsx

@@ -401,7 +401,7 @@ export class AdminCampaignEdit extends React.Component {
           this.props.campaignData.campaign.contactsCount > 0,
         checkSaved: () => !this.state.campaignFormValues.contactData,
         blocksStarting: true,
-        expandAfterCampaignStarts: false,
+        expandAfterCampaignStarts: true,
         expandableBySuperVolunteers: false,
         extraProps: {
           contactsCount: this.props.campaignData.campaign.contactsCount,

src/extensions/contact-loaders/csv-s3-upload/index.js

@@ -128,15 +128,20 @@ export async function processContactLoad(job, maxContacts, organization) {
   const contactsData = (await s3.getObject(params).promise()).Body.toString(
     "utf-8"
   );
-  const contacts = JSON.parse(await gunzip(Buffer.from(contactsData, "base64")))
-    .contacts;
-
+  const parsedData = JSON.parse(
+    await gunzip(Buffer.from(contactsData, "base64"))
+  );
+  const contacts = parsedData.contacts;
   const contactsCount = contacts.length;
+
   await finalizeContactLoad(
     job,
     contacts,
     maxContacts,
     job.payload,
-    JSON.stringify({ finalCount: contactsCount })
+    JSON.stringify({
+      finalCount: contactsCount,
+      filename: parsedData.name || null
+    })
   );
 }

src/extensions/contact-loaders/csv-s3-upload/react-component.js

@@ -92,7 +92,12 @@ export class CampaignContactsForm extends React.Component {
           } else if (contacts.length === 0) {
             this.handleUploadError("Upload at least one contact");
           } else if (contacts.length > 0) {
-            this.handleUploadSuccess(validationStats, contacts, customFields);
+            this.handleUploadSuccess(
+              validationStats,
+              contacts,
+              customFields,
+              file
+            );
           }
         },
         { headerTransformer: ensureCamelCaseRequiredHeaders }
@@ -109,7 +114,7 @@ export class CampaignContactsForm extends React.Component {
     });
   }
 
-  handleUploadSuccess(validationStats, contacts, customFields) {
+  handleUploadSuccess(validationStats, contacts, customFields, file) {
     this.setState({
       validationStats,
       customFields,
@@ -118,6 +123,7 @@ export class CampaignContactsForm extends React.Component {
       contactsCount: contacts.length
     });
     const contactCollection = {
+      name: file.name || null,
       contactsCount: contacts.length,
       customFields,
       contacts
@@ -259,6 +265,20 @@ export class CampaignContactsForm extends React.Component {
   }
 
   render() {
+    if (this.props.campaignIsStarted) {
+      let data;
+      try {
+        data = JSON.parse(
+          (this.props.lastResult && this.props.lastResult.result) || "{}"
+        );
+      } catch (err) {
+        return null;
+      }
+      return data && data.filename ? (
+        <div>Filename: {data.filename}</div>
+      ) : null;
+    }
+
     let subtitle = (
       <span>
         Your upload file should be in CSV format with column headings in the
@@ -286,6 +306,8 @@ export class CampaignContactsForm extends React.Component {
   }
 }
 
+CampaignContactsForm.prototype.renderAfterStart = true;
+
 CampaignContactsForm.propTypes = {
   onChange: type.func,
   onSubmit: type.func,
@@ -297,5 +319,6 @@ CampaignContactsForm.propTypes = {
   saveLabel: type.string,
 
   clientChoiceData: type.string,
+  lastResult: type.object,
   jobResultMessage: type.string
 };

src/extensions/contact-loaders/csv-upload/index.js

@@ -84,12 +84,17 @@ export async function processContactLoad(job, maxContacts, organization) {
   /// * Error handling
   /// * "Request of Doom" scenarios -- queries or jobs too big to complete
 
-  let contacts;
+  let payload;
   if (job.payload[0] === "{") {
-    contacts = JSON.parse(job.payload).contacts;
+    payload = JSON.parse(job.payload);
   } else {
-    contacts = (await unzipPayload(job)).contacts;
+    payload = await unzipPayload(job);
   }
-
-  await finalizeContactLoad(job, contacts, maxContacts);
+  await finalizeContactLoad(
+    job,
+    payload.contacts,
+    maxContacts,
+    null,
+    JSON.stringify({ filename: payload.name || null })
+  );
 }

src/extensions/contact-loaders/csv-upload/react-component.js

@@ -88,6 +88,7 @@ export class CampaignContactsForm extends React.Component {
 
     event.preventDefault();
     const file = event.target.files[0];
+    console.log("file upload", file);
     this.setState({ uploading: true }, () => {
       parseCSV(
         file,
@@ -103,7 +104,12 @@ export class CampaignContactsForm extends React.Component {
               ).toLocaleString()} contacts max – your file contains ${contacts.length.toLocaleString()}.`
             );
           } else if (contacts.length > 0) {
-            this.handleUploadSuccess(validationStats, contacts, customFields);
+            this.handleUploadSuccess(
+              validationStats,
+              contacts,
+              customFields,
+              file
+            );
           }
         },
         { headerTransformer: ensureCamelCaseRequiredHeaders }
@@ -120,7 +126,7 @@ export class CampaignContactsForm extends React.Component {
     });
   }
 
-  handleUploadSuccess(validationStats, contacts, customFields) {
+  handleUploadSuccess(validationStats, contacts, customFields, file) {
     this.setState({
       validationStats,
       customFields,
@@ -129,6 +135,7 @@ export class CampaignContactsForm extends React.Component {
       contactsCount: contacts.length
     });
     const contactCollection = {
+      name: file.name || null,
       contactsCount: contacts.length,
       customFields,
       contacts
@@ -270,6 +277,19 @@ export class CampaignContactsForm extends React.Component {
   }
 
   render() {
+    if (this.props.campaignIsStarted) {
+      let data;
+      try {
+        data = JSON.parse(
+          (this.props.lastResult && this.props.lastResult.result) || "{}"
+        );
+      } catch (err) {
+        return null;
+      }
+      return data && data.filename ? (
+        <div>Filename: {data.filename}</div>
+      ) : null;
+    }
     let subtitle = (
       <span>
         Your upload file should be in CSV format with column headings in the
@@ -297,6 +317,8 @@ export class CampaignContactsForm extends React.Component {
   }
 }
 
+CampaignContactsForm.prototype.renderAfterStart = true;
+
 CampaignContactsForm.propTypes = {
   onChange: type.func,
   onSubmit: type.func,
@@ -309,6 +331,7 @@ CampaignContactsForm.propTypes = {
 
   clientChoiceData: type.string,
   jobResultMessage: type.string,
+  lastResult: type.object,
 
   maxNumbersPerCampaign: type.number,
   contactsPerPhoneNumber: type.number