https://github.com/levye/go-appmetadata-yaml/blob/master/README.md
REST API to support read/write application metadata as yaml/json payloads with a integrated in-mem db
- Worker Thread-Pools request processing with channels
- Async Logger using Channels
- Custom validation function integration to handler
- GET and POST to create and get application metadata
- yaml and json payload support
- Acecept header support (application/json) default is yaml
- Logger and Validator middlewares
Project structure:
-
This is where main function is defined and all the components are being utilized and integrated. main func simple performs;
- create async logger
- create in-memory storage for application metadata
- create application contex to access common functionality across different modules
- initialize work queue and dispatcher
- create server which also initializes handlers
- listen and serve
(to see how these steps are being implemented, continue reading) Sample usage of the server is:
-
Async logger implementation on top of go logger utilizing Go channels. It provides three level of logging which is
-Warning (stdout) -Info (stdout) -Error (stderr) -Fatal (stderr)For each level, a log object and a channel
Sample usage for logger is :
asyncLogger := logger.CreateAsyncLogger() asyncLogger.Log(logger.INFO, "log message..") asyncLogger.Log(logger.WARNING, "log message..") asyncLogger.Log(logger.ERROR, "log message..") asyncLogger.Log(logger.FATAL, "log message..")
It is a simple in-memory strorage to store application metadata Supports Insert and Read methods.
type Database interface { Insert(key string, val interface{}) Read(key string) interface{} ReadWithParams(params map[string][]string) []interface{} }
This is where we define our Application metadata model. There is no business logic there but only data model itself.
This is where actual server implementation exists. We use chaning logic for handler by implementing Adapter (Decorator) pattern. So that we can add dynamic behaviour without code replication to our handler. In order to do that, we use a middleware signature
which looks like:type middleware func(h http.HandlerFunc) http.HandlerFunc
Bacically, we are getting a HandlerFunc, we are adding extra behaviour it and returning a new HandlerFunc
which enables us to be able to chain multiple HandlerFunc.Let see one of the example about how to use it:
func (s *Server) withLog() middleware { s.Log.LogInfo("withLog called") return func(h http.HandlerFunc) http.HandlerFunc { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { s.Log.LogInfo("Log Before") defer s.Log.LogInfo("Log After") h(w, r) }) } }
As you can see, we are adding logging capability to our HandlerFunc and inside it, we are calling next HandlerFunc in chain.
If you think about the usage of this kind of chaining mechanishm:- You can add logging capabilitiy
- You can perform validation
- You can decide to call or not to call next handlerFunc depending on your logic
- ...
It is also practicle to chain all your middleware function so code will be more readable.
func (s *Server) Chain(h http.HandlerFunc, m ...middleware) http.HandlerFunc
So how we acctually use them, see example below,
type middleware func(h http.HandlerFunc) http.HandlerFunc func (s *Server) createAppMetadataHandler(w http.ResponseWriter, r *http.Request) func (s *Server) withValidation(validator validatorFunc) middleware func (s *Server) searchAppMetadataHandler(w http.ResponseWriter, r *http.Request) //actual handler s.Routers.HandleFunc("/api/v1/apps", s.Chain(s.createAppMetadataHandler, s.withValidation(validator.ValidateRequest), s.withLog())).Methods("POST")
The important point here how we chain and use them,
s.Chain(s.createAppMetadataHandler, s.withValidation(validator.ValidateRequest), s.withLog())).Methods("POST")
This is a cleaer code to read and implement. So whatever behavior you would like to add to your handler,
you can implement it as middleware and chain with other middleware functions.This is a custom validation function to use for handler where we validate incoming request.
We have a predefined signature for validator functions so our validate func should have
exactly same signature in order to use them. The signature is :type validatorFunc func(h *http.Request) (bool, string)
Where it takes request as argumant, which we will validate, and return bool and string where bool indicates that
if validation is successfull or not, and error string that shows where our validation has failed. (error message)
Sample usage of the server is:
//create async logger
asyncLogger := logger.CreateAsyncLogger()
storage := memstore.CreateInMemDB()
storage.SetLogger(asyncLogger)
//create application context
appContext := context.AppContext{
Storage: storage,
Logger: asyncLogger,
}
//initialize dispatcher and pools
workQueue := make(chan workpool.WorkRequest, MaxQueue)
dispatcher := workpool.NewDispatcher(workQueue, MaxWorker, &appContext)
dispatcher.StartDispatcher()
//create server
server := server.CreateServer(&appContext, workQueue)
http.Handle("/", server.Routers)
asyncLogger.Log(logger.INFO, "Listening localhost 8080...")
err := http.ListenAndServe("localhost:8080", server.Routers)
if err != nil {
asyncLogger.Log(logger.FATAL, err.Error())
}
```
Sample application metadata payload is :
```yaml
title: My valid app
version: 1.0.8
company: Ecaglar Inc.
website: https://ecaglar.net
source: https://github.com/levye/repo
license: Apache-2.1
maintainers:
- name: Firstname Lastname
email: emre@hotmail.com
description: |
### blob of markdown
More markdownServer provides a simple enpoint for GET and POST operations.
/api/v1/apps
POST operation is used to create application metadata.
Yaml or json payload is supported as payload and content type is "application/json"
GET operation is used to read application metadata records. Search parameters are being passed via URL. Supported URL query search parameters are:
- { version, title, company, website, source, license, maintainers.name, maintainers.email and description}
Note that title and description parameters are used to check if record contains! those parameters. So full string check does not happen.
Creates a application metadata. Accepts yaml or json payload. Both formats are supported. Since yaml is a superset of
json, yaml parser can also handle json. All fields and valid email addresses are required otherwise returns error.
In order to optimize workload of server and decrease latency, work thread-pool paradigm has been implemented which is natively supported by Go thanks to goroutines, channels and overall native concurrency support of the language.
Summery of the approach is:
Sample POST request is:
http://localhost:8080/api/v1/apps
and body payload is:
title: My valid app
version: 1.0.8
company: Ecaglar Inc.
website: https://ecaglar.net
source: https://github.com/levye/repo
license: Apache-2.1
maintainers:
- name: Firstname Lastname
email: emre@hotmail.com
description: |
### blob of markdown
More markdownGET operation also has same endpoint. Changing the URL query parameters, you can query different records.
POST - /api/v1/apps
Accepts application metadata payload in yaml or json format in body.(application/son)
GET - /api/v1/apps
Returns all records
GET - /api/v1/apps?version=1.0.0
Returns the record with version 1.0.0
GET - /api/v1/apps?version=1.0.0&title=my%20app
Returns the record with version 1.0.0 if exists.Does not check other parameters as version number is unique.
GET - /api/v1/apps?company=mycompany.com&title=my%20app
Returns record(s) with company name "mycompany.com" and title contains "my app"
GET - /api/v1/apps?description=latest
Returns record(s) with description contains "latest"
GET - /api/v1/apps?maintainers.name=Bill&maintainers.name=Joe
Returns record(s) which have/has maintainers name "Bill" and "Joe"
GET - /api/v1/apps?maintainers.email=bill@c.com&license=Apache-2.1
Returns record(s) which have/has maintainers email "bill@hotmail.com" with licence "Apache-2.1"
TO GET PDF VERSION OF README.md