Pluggable extensions

[jmattheis]

It would be nice to have pluggable extensions for Gotify. This could f.ex. be used for polling the GitHub Notification API.

• Golang Plugin API or some kind of JavaScript engine to enable pluggable code
• ? Allow adding routes something like /plugin/:name/... (overkill ?!?)
  • Could be useful for webhooks
• ? Users can enable plugins
• ? Key-Value per user for configuration (f.ex authentication token)

[eternal-flame-AD]

I have implemented a basic plugin support here. Featuring:

• Plugin Scheme Version Verification
• Internal requests
• Register handlers

Here is a collection of plugins that I wrote.

Currently, I use it to receive webbooks from netlify and it worked perfectly. I think further discussion on how the plugin could be implemented could be made. Maybe we could implement:

• External Authentication source(System authentication #78)
• Stream processing?

[jmattheis]

@eternal-flame-AD I think this is a step in the right direction, but currently it looks pretty hacky (f.ex. using the httptest package for building the request). Also I feel like your use case with netify doesn't really need plugin support, you could build a standalone executable with a simple http interface which then pushes messages to gotify/server via the rest api.

I would only see benefits in the plugin api if it would support multi users. This means Gotify must hold a configuration per user for a plugin, Example: Rss-Feed Urls, GitHub authentication token.
This configuration would probably be a simple key value store. Plugins would specify what config they need and Gotify would dynamically create a config UI per plugin.

[eternal-flame-AD]

Internal requests

currently it looks pretty hacky (f.ex. using the httptest package for building the request)

Yeah this is not very elegant and I think there should be a better way to do this. I think we could build a official golang application API wrapper and use the official wrapper to provide APIs for plugins, eg:

// package application is the official gotify applications API wrapper
package applications

type GotifyCommunicator interface {
	SendMsg(token string, msg Message) (*MessageResult, error)
	Version() (*Version, error)
	// More methods on gotify APIs
}

// Remote Application is used to make remote API calls to gotify, implements GotifyCommunicator
type RemoteApplication struct {
	Endpoint string
}

func (c RemoteApplication) Version() (*Version, error) {
	req, err := http.NewRequest("GET", c.Endpoint+"/version", nil)
	// The rest of the implementation
}

// more implementations...

package myplugin

var Engine *applications.GotifyCommunicator

func getVersion() {
    v, err := Engine.Version()
    // ...
}

Whether to build a plugin

your use case with netify doesn't really need plugin support

Literally, any plugin could be implemented by building a standabone executable and proxying requests to gotify. IMO the reason we needed plugins is that we do not want our hosting server to be running all kinds of processes and maintaining different configurations just to receive different webhooks from different upstream servers, translate them into gotify format and send it to the gotify endpoint. An official plugin support for webhook proxies could help build a unified configuration scheme between all peripheral projects just as you have pointed out on the config UI thing. IDK how many users are currently using gotify to handle webhooks, if it is not minority then I think we should have built-in support for this, even though it could be implemented without plugins.

Configuration

This configuration would probably be a simple key value store. Plugins would specify what config they need and Gotify would dynamically create a config UI per plugin.

There are three kinds of implementation of plugin configuration I have thought of by now:

1. Add a plugins key to the main configuration file, like this:

plugins:
  - netlify.so:
      - jwt: abc123
      - apptoken: A1234
  - user_auth.so:
      - users: ["root","user"]
  - rss.so:
      - checkinterval: 30m
      - reportonerror: true
      - apptoken: A5678

This provides a more unified configuration experience with the main gotify program. But it would provide less options for those who prefer configuration with the WebUI

2. Use a simple key-value configuration, eg the implementation might look like this, the ui would inspect Config with reflection and generate corresponding UI interfaces for configuration:

// plugins is a subpackage in gotify
package plugins

type AppTokenSelector string
func (c AppTokenSelector) Verify(value interface{}) error {
    return tokenIsValid(c.(string))
}

type Verifier interface {
    Verify(value interface{}) error
}

package myplugin

type RepoName string

func (c RepoName) Verify(value interface{}) error {
	return repoExist(value.(string))
}

var Config struct {
	GHUserName   string                   `default:"jmattheis" plugins:"github_user_name"`
	Repo         RepoName                 `plugins:"repository_name"`
	AppToken     plugins.AppTokenSelector `plugins:"application_token"`
	PollInterval int                      `plugins:"poll_interval"`
}

func (c Config) Verify(value interface{}) error {
	config := value.(Config)
	return configContextIsValid(config)
}

This is more in favor of the WebUI. However it would be difficult to maintain a structured format for those who prefer configuring with configuration files or environment variables.

3. Use a similar approach to how gotify manages configurations by placing a $PLUGIN_NAME.yml just beside config.yml or env GOTIFY_$PLUGIN_NAME_$CONFIG_KEY.

This provides a config experience most similar to how gotify itself is configured. However we would have a bunch of extra config files and/or env variables.

Persistent Storage

Actually I have another concern is how to provide persistent storage to plugins. Currently I only have this idea:

package myplugin

// Filled in on plugin initialization, serialize PersistentStorage and save in database?
var SavePersistentStorage func()

// Filled in on plugin initialization
var PersistentStorage struct {
    SessionKey string
}

func authWithRemote() {
   key := getSessionKey()
   PersistentStorage.SessionKey = key
   SavePersistentStorage()
}

[eternal-flame-AD]

@jmattheis I hoped to listen from your opinion on:

• What is the functionality boundary for plugins
• How the configuration could be implemented, one of the three I mentioned in the last comment or some other way?
• Should we provide a persistent storage API and in which way?

[jmattheis]

@eternal-flame-AD thanks for sharing your thoughts (:.

This is just a brain-dumb. So nothing is final or even really thought out.

• If we have plugins for multiple users the best way would be to store the config in a database.

  type UserConfig struct {
       UserID uint
       PluginID string?!?!?
       Value string // could be json/yaml
  }

• Dynamically generating a UI seems pretty expensive. So maybe we could give the user a yaml editor in the gotify ui which supports validation/autocompletion and plugins then provide a schema for the config. (https://github.com/pengx17/monaco-yaml)
• A user should be able to disable and enable plugins for himself
• Each user could have an own instance of a plugin, that would simplify things. An application could be automatically created per plugin then the communicator interface would be simpler.

  type Communicator interface {
      SendMessage(*model.Message) error
  }
  type UserContext struct {
     Communicator Communicator
     UserStore UserStorage 
     User model.User
  }
  // UserStorage saves values the serialized value in the db and deserialized it in get.
  type UserStorage interface {
       Save(interface)
       Get(interface{}) // pass by reference 
  }
  
  type Plugin interface {
        OnConfigChange(*UserContext) error
        Disable()
  } 

  Each plugin implementation then would have a new method which returns a plugin like this:

  func New(*UserContext) Plugin {
  }

• each plugin could have a sub router(mux) which can be dynamically changed when a user enables/disables the plugin.
• the api wrapper for the plugins should not make requests to the engine from gotify. Something like sending messages should be extracted that it can be used by the plugin api and the rest api
• using config files for plugins seems not very user friendly. Gotify's global config is a file but user specific things should be saved in the database.
• for the first plugin-api I would only support adding routes and sending messages

[eternal-flame-AD]

User Context

I highly agree with you about how plugins should be initialized with a UserContext. Users would have a private plugin zone at /plugins/:pluginName/:userID/

Subrouter

I currently consider exposing these endpoints:

/plugins/:pluginName/_config/schema

// return config schema

/plugins/:pluginName/:userID/<extra_path>

// maintained by plugins, custom router mux

/plugins/pluginName/:userID/_state

// diable or enable plugin

/plugins/pluginName/:userID/_config

// get or update user configuration

Plugin types

I think we could add more interfaces in the future to provide more functionality to plugins while keeping backward capability, eg:

// plugin base, all plugins should statisfy this
type Plugin interface {
      Start() error
      Stop() error
}

type Configuor interface {
    Plugin
    ConfigScheme() ConfigScheme
    OnConfig(c []byte) error
}

type Authenticator interface {
    Plugin
    Authenticate(c *gin.Context) (bool, error)
}

type Webhooker interface {
    Plugin
    RegisterHandlers(r *HandlerRegisterRequest) error
}

type NotificationSender interface {
    Plugin
    RegisterSender(s *MessageSender) error
}

type Storager interface {
    Plugin
    GetPersistentStorageHandler(s PersistentStorage) error
}

and so on...
Gotify would try if a plugin satisfies each interface and initialize them on plugin initialization

[jmattheis]

Users would have a private plugin zone at /plugins/:pluginName/:userID/

Maybe replace the userId with a random string to provide some "authentication". (Yeah it still can be bruteforeced)

Apis: I would prefer naming it plugin that it's consistent with other apis.

• GET /plugin - returns all plugins enabled and disabled for the current user
• POST /plugin/:name/enable - enables a plugin for the current user
• POST /plugin/:name/disable - disables a plugin for the current user
• GET /plugin/:name/schema - returns the plugin schema
• POST /plugin/:name/config - sets the config for the current user
• GET /plugin/:name/config - gets the config for the current user
• /plugin/:name/custom/:randUserToken/* - the * path is available for the plugin to configure
• maybe even provide a global scope for the plugin to configure /plugin/:name/global/*

Plugin Types: Yeah we could do this, tho I think I would only support webhooks and sending messages.

[eternal-flame-AD]

Token

I think it is good, but maybe call the random string something like "plugin token" that starts with "P" for consistency?

Plugin types

IMO we definitely should do this, otherwise, it would be painful for future API changes. Plugins could be initialized in this way, very elegant and maintainable in my opinion:

1. Type assert it to Plugin or panic.
2. If it implements Configer, allow this plugin to be configured
3. If it implements Webhooker, pass in the subrouter and let the plugin register its own handlers.
4. If it implements Storager, pass in a persistent storage handler
5. If it implements NotificationSender, pass in a SendMsg hook
6. and so on...

[jmattheis]

I think it is good, but maybe call the random string something like "plugin token" that starts with "P" for consistency?

Yes, good idea.

Plugin types: The structure is okay.

[eternal-flame-AD]

Okay I think there we have reached an agreement on the design.

Would you like it to implement it on your own, or I open a PR for this?

[jmattheis]

@eternal-flame-AD If you like to do it, feel free to do so (:. How about I create a branch on this repo where you can add PullRequests from your fork, then there can be smaller pull requests which can be discussed easier.