#168 add initial devops guidelines

docs/devops-guidelines/.order

@@ -0,0 +1,8 @@
+DevOps-Guidelines
+Plan
+Create
+Verify
+Deploy
+Configuration-Management
+Monitoring
+Package

docs/devops-guidelines/Configuration-Management.md

@@ -0,0 +1,36 @@
+[DevOps Strategy](/Project-Artifacts/DevOps-Strategy)
+
+**Capabilities Inventory**
+
+|Capability|Ideal State|Tools In Inventory|
+|--|--|--|
+|Infrastructure Provisioning|Infrastructure is provisioned and configured as code along with the rest of the workload in an automated CI/CD pipeline|Terraform, AWS CloudFormation, Azure Resource Manager|
+|Infrastructure Configuration|Infrastructure is provisioned and configured as code along with the rest of the workload in an automated CI/CD pipeline|Terraform, AWS Config, AWS CLI, Azure Policy, Azure CLI|
+|Application Provisioning|Provision via Continuous Deployment pipeline using GitHub Actions, Azure Pipelines, or similar; users and administrators provisioned with proper access|Azure DevOps, GitHub|
+|Application Configuration|Configuration settings are exposed based on user roles.  Changes to configuration items are audited and sent through and appropriate approval process.||
+
+**Infrastructure Provisioning**
+- Infrastructure will be provisioned using Terraform templates.  This will allow the team to standardize on a tool several team members already know that the company is licensed for, is supported for common cloud platforms and provides provisiong of most native capabilities.
+
+- All provisioning templates regardless of technology will be stored in git repos and deployed use the pipeline toofl of choice such as GitHub Actions or Azure DevOps Pipelines.
+
+- Infrastructure provisioning should follow [resource naming guidelines](todo ) and [tagging guidelines](todo ) .
+
+**Infrastructure Configuration**
+- Since all resources will be deployed and configured via code (Terraform, CloudFormation, etc.), configurations will be stored in version control (Git Repo).
+
+- Any updates will be reviewed via a Pull Request, then can be deployed by the service of choice. Terraform has the capability to update existing resources or replace resources that cannot be updated in place.
+
+It is critical that any resource that maintains state (databases, blob storage, aka a stateful resource) be backed up and restorable in the event of a resource replacement, or, stateful resources should be excluded from full automated provisioning.
+
+**Application Provisioning**
+- Applications should be provisioned via automated pipelines
+- Pipeline templates will be stored in git repos along with the the application code
+
+**Application Configuration**
+- Non secret application configuration information should be stored with source code.
+
+- Sensitive information, passwords or API keys, should be stored in a cloud native secret store, or, the pipeline tool of choice.
+
+- Secrets based only on deployment environment can be 
+

docs/devops-guidelines/Create.md

@@ -0,0 +1,63 @@
+[[_TOC_]]
+
+Specific tools used to create the source and related artifacts will be determined by a project team.
+
+These guidelines are intended to be general guidance and will be applicable to most teams.
+
+# Managing Artifacts
+For technical and software projects, a variety of artifacts are created. All artifacts should be maintined in git based repos 
+
+- Design artifacts such as high level designs, information architecutre, wire frames and similar should be committed to source and associated with a User Story whenever possible
+- Software developers should create code in a branch created from a User Story or Bug/Defect
+
+todo: Branching Strategies
+todo: Repository Structure
+
+**Capabilities Inventory**
+
+|Capability|Ideal State|Typical Tools|
+|--|--|--|
+|Design|Designs are driven by user stories, captured ahead of development and vetted against a Definition of Ready (DoR) for design acceptance.  Design documents incorporate all aspects of the design, to include at a minimum UX/UI, infrastructure, data models, logic and integration. Design may optionally include a clickable prototype or a proof of concept.|Adobe XD, Zeplin, Adobe Photoshop, Adobe Illustrator|
+|Implementation|Developers work on tasks tied to user stories.  Unit tests are developed for acceptance criteria defined in the design ahead of development of the actual product.  Tasks are completed in priority order.  Development is accelerated by the use of IDEs and feedback tools.|Visual Studio, Visual Studio Code, MuleSoft, SQL Server Management Studio, Oracle SQL Developer, PL/SQL Developer, Resharper
+|Version Control|All artifacts are versioned and revisions to those artifacts are tracked by a version control system (git).  Merges are simple and reversion to a "known-good state" is easy.  The version control system supports distributed development and forms the backbone for the build/deployment pipelines.|GitHub, git based Azure DevOps Repos, et.al. |
+
+# Design
+Designs typically consist of wireframes, mockups or prototypes. 
+- A wireframe is a low-fidelity visual representation of the site and will be used to depict the basic UI, detailing the layout and structure of the site. 
+- A mockup is a mid to high-fidelity visual representation of the site and will be used to highlight color scheme, visual style and typography. 
+- A prototype is a high-fidelity visual representation of the site and will be used to simulate user interaction by allowing the user to experience the content through clickable elements and interactions. 
+
+## Design Guidelines
+Use the design that meets your needs. We recommend starting with wireframes, using mockups and prototypes when necessary and their value is clear. 
+
+- Designs should utilize an existing design system such as Material Design, as well as standardized componenents such as Font Awesome Icons and custom components.
+- Do not develop eponymous designs unless they are truly required for differentiation. 
+- Beware existing or complementary solutions.
+- Regardless of design, usability should not be sacrified: value something that works well over somthing that looks pretty
+- When possible, include user testing in design review. 
+
+Designs should be reviewed by stakeholders and users, as well as the design team, development team and business analysts before finalization. User stories, acceptance criteria and UX personas are used to approve and validate designs.
+
+# Implementation
+Implementation is where the rubber meets the road.  The plans are laid.  The user stories are written.  The test cases have been identified and it's finally time to build something.
+
+Implementation will follow the SCRUM methodology and utilize Test Driven Development (TDD).  All features will be logged as user stories with acceptance criteria before they are worked.
+
+All development is expected to be performed using the appropriate development environmetn such as Visual Studio, Visual Studio Code, Eclipse, ...
+
+## Source Management
+Git repos in Azure DevOps are preferred. 
+
+Associate commits using user stories using the syntax "#<work item number> ..." in commit messsages to track which user story the commit is for. This generates a source audit trail and can be used by release systems to track which stories are being delivered in a release.
+
+Code should be committed on a regular basis. More frequent commits provide benefits:
+- Team members can merge committed changes more often reducing overall merge effort
+- Continous integration builds can be triggered to verify code when it is committed
+
+## Coding Standards
+Developers will follow the appropriate coding standards for the languages being used.  Coding standards can be found at 
+
+## Unit Tests
+Unit tests will be developed for each user story that comply with the and are adequate to test the feature.
+
+

docs/devops-guidelines/Create/Repository-Structure.md

@@ -0,0 +1,104 @@
+---
+
+# Introduction
+
+One of the bigger challenges to building software is managing the lifecycle of the source code, from authorship to deployment. Below is an outline of how to structure our source code, the goal of which is to achieve the following:
+
+- Any change must be easily reviewable, including how the change affects dependencies.
+- Building and deploying should be simple, fast, and reproducible.
+- Changes should be testable in both isolation and integrated with the existing environment.
+- Setting up new users/systems should require as few steps as possible and require little outside assistance.
+
+---
+
+# Review
+
+There are generally two approaches when setting up source control for a project:
+
+- Each application's code remains in its own repository
+- All code is in one repository (monorepo)
+
+Neither implementation is objectively better than the other; they are both highly dependent on the complexity of the application and delivery model. Below is a review of the benefits and challenges that come with each approach. 
+
+
+## <u>Separate Repositories</u>
+
+### Benefits
+
+- Organization  
+The structure of the repo is simple; it contains the source files and configuration from only one project.
+- Isolation  
+Each project exists in isolation. Changes can be made, and subsequently reviewed, knowing they have no direct or immediate impact on anything else.
+
+### Challenges
+
+- Dependencies
+Coordinating changes between multiple, dependent projects is a tedious process. It is made worse over a project's lifecycle as it naturally grows in complexity. One example of this would be a web app that is dependent on a component library. When the app requires a change to a component, the dependent component repo must go through its code review and release process before the web app is able to move forward. If there are further changes required of the updated component library, the cycle starts over and the delay increases.
+- Deployments
+The ability to automate build and release pipelines suffers when the dependency graph grows in complexity. Coordinating deployments requires a significant effort from the team to track interdependent versions and where they should exist in the release cycle.
+
+
+## <u>Single Repository (monorepo)</u>
+
+### Benefits
+
+- Organization
+All files exist in one, well-structured directory. Users can easily search across projects without having to sync updates and branches across multiple repositories.
+- Dependencies
+Every project moves forward together. Changes in dependencies should never block progress since review cycles and upstream integration never happen orthogonally.
+- Tooling
+Most applications and services operate on the file system. Providing immediate access to all required files greatly improves the speed, efficiency, and resiliency of the CI/CD pipelines.
+
+### Challenges
+
+- Scalability
+The life of a repository is long-lived. The VCS index continues to grow with every new file or changed line. While there are ways to mitigate this, the size of the repo will inevitably grow.
+
+---
+
+# Model Repository Implementation
+
+Our project is comprised of multiple slices of related functionality and will be developed in parallel by a distributed team. The current approach will maintain a single repository ("monorepo") containing top-level directories for each independent project.
+
+The structure of the repository will be similar to the following example:
+
+- <u>**Root**</u> (repo directory)
+  - Repo-level config files: e.g. `.gitignore`, `.gitattributes`, `.editorconfig`
+  - Makefile  (or similar) to provide top-level hooks into available scripts
+  - <u>**docs**</u>
+    - README.md
+    - installation.md
+  - <u>**scripts**</u> (utilities for cross-project coordination)
+    - README.md
+    - Makefile
+    - pipeline
+      - build.sh
+      - test.sh
+    - utils
+      - setup_machine.sh
+  - <u>**infrastructure**</u>
+    - README.md
+    - Makefile
+    - env
+      - ring1
+      - ring2
+    - modules
+      - lambda
+      - s3
+  - <u>**frontend**</u>
+    - Project-level config files: e.g. `.gitignore`, `.npmrc`, `setup.cfg`
+    - docs
+      - README.md
+      - installation.md
+    - scripts
+      - build.sh
+      - test.sh
+    - src
+    - tests
+  - <u>**api**</u>
+  - <u>**shared_components**</u>
+  - <u>**bloomreach**</u>
+
+
+---
+

docs/devops-guidelines/Deploy.md

@@ -0,0 +1,24 @@
+[[_TOC_]]
+
+Project code and artifacts are built locally using tools dictated by the software architecture of a project.
+
+Regardless of architecture, projects should automated builds to support *Continuous Integration* or *CI*, and, automated deployment to staged environments
+
+# Continuous Integration
+Build the solution on every commit. Build should inluding executing and passing all unit tests as well as security code scans.
+
+# Continous Deployment
+When software is succesfully built, it should be deployed immediately. It is most common to deploy in stages so that deploymnets can be verified and tested.
+
+|Stage | Description |
+|-------|----------|
+|Dev |Deploy to dev on every successful build of the system. Run integration tests.|
+|Test |Proceeds whenever Dev passes all tests. Additional integration tests are performed. In some cases, manual tests or automated UI tests are executed, or, 3rd party test systems are included in integration tests|
+|Pre production|As close to production as reasonable. May have a copy of production data to test migrations on and/or run additional production tests|
+|Production|final production environment or environments | 
+
+## Ring Deployment Model
+
+Often, deployment to Production isn't deployment to a single instance of a system. when multiple systems exist, supporting multiple regions around the world for example, a ringed deployment should be used. This allows for controlled rollouts and limits the impact of deployment issues only discovered as 
+
+Supports rolling and red/green deployments.

docs/devops-guidelines/DevOps-Guidelines.md

@@ -0,0 +1,20 @@
+# DevOps Approach
+
+"DevOps is cultural fusion of People, Process, and Technology to accelerate delivery of Business Value"
+
+![DevOps Continous Improvement](.images/DevOps.png)
+
+A complete DevOps approach combines critial develop, operations and management concerns so that software projects can continously and consistently deliver high quality solutions. 
+
+Each practice within these guidelines works with others to form a working model for continous improvement. In addition, DevOps works hand in hand with portfolio and project management and agile initiatives.
+
+This wiki is intended for the practitioners: team members on projects. It  provides a set of guidelines that can be used as a starting point for any project's DevOps implementation. Each project will need to define specific tools and processes that will work best for that project. This wiki can be cloned and used as a starting point for a project specific DevOps Strategy.
+
+Topics covered in this site include:
+- [Planning and managing work from inception to delivery](/Plan)
+- [Source management](/Create)
+- [Verifying quality](/Verify)
+- [Build and release automation](/Deploy)
+- [Monitoring a solution in production](/Monitor)
+
+This site assumes you are using and are familiar with [GitHub](https://github.com) or [Azure DevOps](https://dev.azure.com).

docs/devops-guidelines/Monitor.md

@@ -0,0 +1,26 @@
+[DevOps Strategy](/Project-Artifacts/DevOps-Strategy)
+
+**Capabilities Inventory**
+
+|Capability|Ideal State|Tools In Inventory|
+|--|--|--|
+|Performance Monitoring|Well established baselines and KPIs that can be tested against, monitored, and improved as a part of CI/CD pipeline and release cycles|AWS CloudWatch, Microsoft SCOM|
+|Security Monitoring|Automatically monitor and alert for appropriate credential rotation events, critical vulnerability patches, and user activity anomalies|AWS CloudTrail, SkyHigh, Dome9, QRadar|
+|Feedback Management|Build feedback submission tools into user interfaces to obtain feedback directly from customers; leverage Product Owner to translate feedback into feature-oriented user stories to be captured and prioritized by planning tools for future releases|Azure DevOps|
+|Consumption Management|Utilize monitoring tools to visualize and report consumption metrics including costs. Use standard resource tags to define and aggregate costs|Azure Monitor, AWS CloudHealth, Splunk, ...|
+|Telemetry|Data should be captured for all of the major types of telemetry data, business metrics, application metrics, infrastructure metrics and deployment pipeline metrics.  Telemetry data is analyzed and leveraged as KPIs to improve the product as part of the feedback loop.| |
+
+**Performance Monitoring**
+todo
+
+**Security Monitoring**
+todo
+
+**Feedback Management**
+todo
+
+**Consumption Management**
+todo
+
+**Telemetry**
+todo

docs/devops-guidelines/Package.md

@@ -0,0 +1,29 @@
+**Capabilities Inventory**
+
+|Capability|Ideal State|Tools In Inventory|
+|--|--|--|
+|Package Configuration|Packages are managed by a package manager which groups software together into a release.  This will include infrastructure, application code and external libraries. All artifacts are tracked by version. Dependencies between various components and versions are tracked to reduce the risk of errors and create a repeatable build process.|Azure DevOps|
+|Approval Workflow|Releases are managed by approvals at each quality gate.  Nothing moves without an approval.  Approval workflow supports delegation and escalation.|Azure DevOps, ServiceNow|
+
+
+**Package Configuration**
+Packages (builds) and deployments will be automated with Azure DevOps Pipelines and assigned an automated version that increments with each build. Azure DevOps creates reliable builds on all platforms through integration with Azure Repos using Git for pull requests, checks, and statuses. Code will be built, tested and deployed via Azure DevOps Repos and Pipelines.
+
+**Approval Workflow**
+
+The approval workflow provided by Azure DevOps will be used for all flows except the final push to production.  
+
+*Deployments to production will require the creation of a change request in ServiceNow.*
+
+The workflow will be as follows:
+
+* Developers can build to their local environment and Ring 1 as needed.
+* Developer initiates pull request.
+* Development Lead and Architect review code and infrastructure.
+* Development Lead and Architect approve code to be merged.
+* Merge triggers a build to Ring 1.
+* All unit tests and automated scans are triggered.
+* If all unit tests and automated scans pass, the Architect and Quality Assurance Lead are notified.
+* Quality Assurance Lead and Architect approve deployments to Ring 2 (Integration/Build Testing).
+* Quality Assurance Lead approves deployments to Ring 3 (Functional Testing) and Ring 4 (UAT).
+* Quality Assurance Lead and Product Owner approve deployment to Ring 5 (Production).