Ingest entities and relationships

docs/development.md

@@ -1,28 +1,74 @@
 # Development
 
-Add details here to give a brief overview of how to work with the provider APIs.
-Please reference any SDKs or API docs used to help build the integration here.
+This integration focuses on
+[Malware Bytes' Nebula](https://www.malwarebytes.com/business/cloud/) and is
+using [Nebula API](https://api.malwarebytes.com/nebula/v1/docs) for interacting
+with the Nebula platform.
 
-## Prerequisites
+## Provider account setup
 
-Supply details about software or tooling (like maybe Docker or Terraform) that
-is needed for development here.
+You can sign up for a trial account by visiting the following
+[link](https://www.malwarebytes.com/business/request_trial/).
 
-Please supply references to documentation that details how to install those
-dependencies here.
+## Authentication
 
-Tools like Node.js and NPM are already covered in the [README](../README.md) so
-don't bother documenting that here.
+To get the client id and client secret you first need to create an API
+key/profile. To do so, login to
+[the cloud platform](https://cloud.malwarebytes.com/), select "Settings" from
+the left side menu, and then click on "APIs & Integrations".
 
-## Provider account setup
+Next click on the "Add" button on the page that just opened, select "Read"
+access, give it a name if you want and then click "Save". The client ID and
+client secret will be visible. Note that this is the only time the secret will
+be visible so it's a perfect opportunity to save it in the env file by following
+steps.
 
-Please provide information about the steps needed to create an account with a
-provider. Images and references to a provider's documentation is very helpful
-for new developers picking up your work.
+1. Create a .env file at the root of this project and set the CLIENT_ID variable
+   to the client ID that you received from the previous step.
 
-## Authentication
+```bash
+CLIENT_ID="client id value"
+```
+
+2. Set the .env's CLIENT_SECRET variable to the client secret that you received
+   from the previous step as well.
+
+```bash
+CLIENT_ID="client id value"
+CLIENT_SECRET="client secret value"
+```
+
+3. Next, you need to find your account id value. One way to find is to look at
+   the URL just after logging in to the cloud platform. When you're on the
+   dashboard page, the URL should look like
+   https://cloud.malwarebytes.com/your-account-id/dashboard. Set the .env's
+   ACCOUNT_ID variable to the account id value you find in the URL.
+
+```bash
+CLIENT_ID="client id value"
+CLIENT_SECRET="client secret value"
+ACCOUNT_ID="account id value (xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx)"
+```
+
+4. Finally, you also need to set two additional .env's variables.
+
+One is MIN_SCANNED_SINCE_DAYS which represents the minimum number of days since
+the last execution time (or now) to use when searching for scanned hosts and
+agents.
+
+Another is MIN_FINDINGS_SINCE_DAYS which represents the minimum number of days
+since the last execution time (or now) to use when searching for findings and
+host detections.
+
+```bash
+CLIENT_ID="client id value"
+CLIENT_SECRET="client secret value"
+ACCOUNT_ID="account id value (xxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx)"
+MIN_SCANNED_SINCE_DAYS="minimum number of days (i.e. 5, 10, 30, etc)"
+MIN_FINDINGS_SINCE_DAYS="minimum number of days (i.e. 5, 10, 30, etc)"
+```
 
-Supply details here for information on how to authenticate with a provider so
-that developers have an idea of what's needed to hit APIs. It may be useful to
-provide explanations for each value specified in
-[../src/instanceConfigFields.json](../src/instanceConfigFields.json).
+After following the above steps, you should now be able to start contributing to
+this integration. The integration will pull in the `INSIGHT_CLIENT_USERNAME`,
+`INSIGHT_CLIENT_PASSWORD` and `INSTANCE_HOSTNAME` variables from the `.env` file
+and use them when making requests.

docs/jupiterone.md

@@ -2,21 +2,9 @@
 
 ## Setup
 
-In this section, please provide details about how to set up the integration with
-JupiterOne. This may require provisioning some resources on the provider's side
-(perhaps a role, app, or api key) and passing information over to JupiterOne.
-
-## Data Model
-
-Provide an overview here of the resources collected from the integration. Please
-provide a mapping of how the resources collected map to the JupiterOne Data
-Model. The tables below were taken from the Azure integration to provide an
-example of how to display that information.
-
-When you start developing an integration, please clear out the tables below. As
-you add support for new entities and relationships, please update the tables and
-document the addition in the [CHANGELOG.md](../CHANGELOG.md) file at the root of
-the project.
+JupiterOne provides a managed integration for Malware Bytes Nebula. The
+integration connects directly to Malware Bytes Nebula API to obtain
+configuration metadata and analyze resource relationships.
 
 <!-- {J1_DOCUMENTATION_MARKER_START} -->
 <!--
@@ -35,19 +23,27 @@ https://github.com/JupiterOne/sdk/blob/master/docs/integrations/development.md
 
 The following entities are created:
 
-| Resources | Entity `_type` | Entity `_class` |
-| --------- | -------------- | --------------- |
-| Account   | `acme_account` | `Account`       |
+| Resources     | Entity `_type`               | Entity `_class` |
+| ------------- | ---------------------------- | --------------- |
+| Account       | `malwarebytes_account`       | `Account`       |
+| Configuration | `malwarebytes_configuration` | `Configuration` |
+| Endpoint      | `user_endpoint`              | `Host`          |
+| Finding       | `malwarebytes_finding`       | `Finding`       |
+| Group         | `malwarebytes_group`         | `Group`         |
+| HostAgent     | `malwarebytes_agent`         | `HostAgent`     |
 
 ### Relationships
 
 The following relationships are created/mapped:
 
-| Source Entity `_type` | Relationship `_class` | Target Entity `_type` |
-| --------------------- | --------------------- | --------------------- |
-| `acme_account`        | **HAS**               | `acme_user`           |
-| `acme_account`        | **HAS**               | `acme_group`          |
-| `acme_group`          | **HAS**               | `acme_user`           |
+| Source Entity `_type`  | Relationship `_class` | Target Entity `_type`        |
+| ---------------------- | --------------------- | ---------------------------- |
+| `malwarebytes_account` | **HAS**               | `malwarebytes_agent`         |
+| `malwarebytes_account` | **HAS**               | `malwarebytes_group`         |
+| `malwarebytes_agent`   | **IDENTIFIED**        | `malwarebytes_finding`       |
+| `malwarebytes_agent`   | **PROTECTS**          | `user_endpoint`              |
+| `malwarebytes_group`   | **HAS**               | `malwarebytes_agent`         |
+| `malwarebytes_group`   | **HAS**               | `malwarebytes_configuration` |
 
 <!--
 ********************************************************************************

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jupiterone/integration-template",
-  "version": "0.0.0",
+  "version": "1.0.0",
   "description": "A JupiterOne Integration",
   "license": "MPL-2.0",
   "main": "dist/index.js",
@@ -24,11 +24,19 @@
     "prepack": "yarn build"
   },
   "peerDependencies": {
-    "@jupiterone/integration-sdk-core": "^3.1.0"
+    "@jupiterone/integration-sdk-core": "^5.0.0"
   },
   "devDependencies": {
-    "@jupiterone/integration-sdk-core": "^3.1.0",
-    "@jupiterone/integration-sdk-dev-tools": "^3.1.0",
-    "@jupiterone/integration-sdk-testing": "^3.1.0"
+    "@jupiterone/integration-sdk-core": "^5.0.0",
+    "@jupiterone/integration-sdk-dev-tools": "^5.0.0",
+    "@jupiterone/integration-sdk-testing": "^5.0.0",
+    "@types/lodash": "^4.14.165",
+    "@types/node-fetch": "^2.5.7",
+    "@types/qs": "^6.9.5"
+  },
+  "dependencies": {
+    "lodash": "^4.17.20",
+    "node-fetch": "^2.6.1",
+    "qs": "^6.9.4"
   }
 }

src/calculateConfig.ts

@@ -0,0 +1,116 @@
+import snakeCase from 'lodash/snakeCase';
+import {
+  IntegrationExecutionContext,
+  IntegrationValidationError,
+} from '@jupiterone/integration-sdk-core';
+
+import {
+  DEFAULT_FINDINGS_SINCE_DAYS,
+  DEFAULT_SCANNED_SINCE_DAYS,
+} from './constants';
+
+import { CalculatedIntegrationConfig, IntegrationConfig } from './types';
+
+export function calculateConfig({
+  instance,
+  executionHistory,
+}: IntegrationExecutionContext<
+  IntegrationConfig
+>): CalculatedIntegrationConfig {
+  const config = instance.config;
+
+  const now = Date.now();
+  const lastSuccessfulExecutionTime =
+    executionHistory.lastSuccessful?.startedOn || 0;
+
+  const minScannedSinceDays = readPropertyAsNumberFromEnvOrConfig(
+    config,
+    'minScannedSinceDays',
+    DEFAULT_SCANNED_SINCE_DAYS,
+  );
+
+  const minScannedSinceTime = sinceDaysTime({
+    now,
+    sinceDays: minScannedSinceDays,
+  });
+
+  const minScannedSinceISODate =
+    lastSuccessfulExecutionTime > minScannedSinceTime
+      ? isoDate(lastSuccessfulExecutionTime)
+      : isoDate(minScannedSinceTime);
+
+  const maxScannedSinceISODate = isoDate(executionHistory.current.startedOn);
+
+  const minFindingsSinceDays = readPropertyAsNumberFromEnvOrConfig(
+    config,
+    'minFindingsSinceDays',
+    DEFAULT_FINDINGS_SINCE_DAYS,
+  );
+
+  const minFindingsSinceTime = sinceDaysTime({
+    now,
+    sinceDays: minFindingsSinceDays,
+  });
+
+  const minFindingsSinceISODate =
+    lastSuccessfulExecutionTime > minFindingsSinceTime
+      ? isoDate(lastSuccessfulExecutionTime)
+      : isoDate(minFindingsSinceTime);
+
+  const maxFindingsSinceISODate = isoDate(executionHistory.current.startedOn);
+
+  return {
+    ...config,
+
+    minScannedSinceDays,
+    minScannedSinceISODate,
+    maxScannedSinceISODate,
+
+    minFindingsSinceDays,
+    minFindingsSinceISODate,
+    maxFindingsSinceISODate,
+  };
+}
+
+function readPropertyFromEnv(propertyName: string): string | undefined {
+  const envName = snakeCase(propertyName).toUpperCase();
+  return process.env[envName];
+}
+
+function parsePropertyAsNumber(
+  propertyName: string,
+  value: string | undefined,
+): number | undefined {
+  if (!value) return undefined;
+
+  const numericValue = Number(value) || undefined;
+  if (!numericValue && !!value.trim()) {
+    throw new IntegrationValidationError(`Invalid ${propertyName}: ${value}`);
+  }
+
+  return numericValue;
+}
+
+function readPropertyAsNumberFromEnvOrConfig(
+  config: IntegrationConfig,
+  propertyName: keyof IntegrationConfig,
+  defaultValue: number,
+): number {
+  const envValue = readPropertyFromEnv(propertyName as string);
+  const configValue = config[propertyName as string];
+  return (
+    parsePropertyAsNumber(propertyName as string, envValue) ||
+    parsePropertyAsNumber(propertyName as string, configValue) ||
+    defaultValue
+  );
+}
+
+const MILLISECONDS_ONE_DAY = 1000 * 60 * 60 * 24;
+
+function sinceDaysTime(input: { now: number; sinceDays: number }): number {
+  return input.now - input.sinceDays * MILLISECONDS_ONE_DAY;
+}
+
+function isoDate(time: number) {
+  return new Date(time).toISOString().replace(/\.\d{1,3}/, '');
+}