Plugin support for extensible automated checks

[grovesy]

The proxy should be extended using plugins to implement automated checks.

This should follow a 'plugin' architecture, where the modules are written outside of the core-product. e.g.

npm install gitproxy-aws-secret-scanner

[coopernetes]

Here's my proposal to split apart git-proxy and improve its modularity. Listing out some known constraints and collecting existing issues that are related to or may be better enabled by implementing this. Let me know if I missed any that are important.

#183
#99
#42
#27
#26

Goals

• allow for git-proxy users to pick and choose the functionality they want. A plugin being removed should not break git-proxy (need to verify certain assumptions in the code)
• enable easy extensibility. People can develop their own internal tooling around git-proxy so long as they fulfill the interface
• support multiple deployment scenarios (Docker, Kubernetes, cloud, FaaS, event-based, etc)

To do

• re-organize & refactor to separate existing features into a discreet set of modules. We can do this using existing or new API boundaries. These APIs will be used by plugin implementers and git-proxy can load them at startup.
• verify if any current assumptions need to be addressed (hard-coded values, etc)
• update tooling to build these new modules.

To avoid

• introducing too many layers of abstraction or indirection, making it difficult to extend & maintain git-proxy
• change any APIs too much once plugins are starting to use them

Below is a sketch of some known modules to consider as candidates for the first set of plugins. Arrows indicate when one plugin may depend on another or which dependencies are shared. Ultimately, I think the proof will be in a working prototype. I'll start a PR for one vertical slice of current of functionality and re-arrange it to make it more of a standalone module or package to be consumed by a plugin such as separating out the server from the proxy. Both should be interchangeable.

Round boxes are shared libraries and square boxes are plugins fulfilling the interfaces of those libraries.

Proposed plugin "interfaces":

• policy check: A plugin which performs a check based on git data (remote, author/committer, is the commit signed, etc)
• policy action: A plugin which performs an action based on a check. Examples include block, log, notifySlack, etc.
• db: A plugin which handles persistence of git-proxy operations and audit events. Examples include in-memory, file, Mongo, Postgres, etc.

---
title: Plugin system proposal
---

flowchart RL

    subgraph git-proxy-core
        direction LR
        git([git protocol])
        check([policy check])
        action([policy action])
        db([db/sink])
        inmem(in memory, file) -.-> db
        auth([user auth])
        cfg([config])
        file(local file for config/auth) --> auth
        file --> cfg
    end

    server(git-proxy-server) -.-> git-proxy-core

    awssecrets(git-proxy-aws-secret-scanner) --> git-proxy-core
    renovate(git-proxy-renovate-scanner) --> git-proxy-core
    xray(git-proxy-xray-scanner) --> git-proxy-core

    mongo(git-proxy-mongodb) --> git-proxy-core
    psql(git-proxy-postgresql) --> git-proxy-core
    redis(git-proxy-redis) --> git-proxy-core
    console(git-proxy-mgmt-console) -.-> server
    opa(git-proxy-opa) --> git-proxy-core
    hashisent(git-proxy-hashi-sentinel) --> git-proxy-core
    ldap(git-proxy-auth-ldap) --> git-proxy-core
    aad(git-proxy-auth-azuread) --> git-proxy-core

Loading

[coopernetes]

@JamieSlome any desire to convert git-proxy to TypeScript? I realized that implementing a plugin system without interfaces will be challenging for plugin authors.

[JamieSlome]

@coopernetes - Yes, we should spin up an issue for this 👍 It will be better practice for our development of the library, as well as for anyone else looking to contribute or build plugins.

Can we spin up an issue to introduce TypeScript to the project with linting support?

[coopernetes]

I'm marking this as done as part of #269 #539