Election rules implementation

[sparshev]

The election rules right now are quite simple:

• Fish is not busy with another application
• Fish is asking Application & Label specified driver on serving the required resources

This answers "yep" or "nope" to the Application. Based on those parameters and the "last resort" random number the cluster choose one node to execute the Application. Or if all the nodes are busy (answering "Nope") - than goes to the next election round and repeats the process.

But there is much more variables to consider - for example driver can check the images size to download from Artifactory and calculate how much time it will take to unpack them and actually run the VM - so pre-cached images will boost the startup time for sure and this node should be used in priority. Or the application wants to use some specific machine (with particular IP address) to serve some security needs.

So all of that could be defined as Rules engine inside the cluster, and cluster or the application can choose which kind of rules to use to detect the winner of the election. Go templates could be used here to define the logic.

The available resources (CPU, Mem, Disk) need to be considered during making decision and overcommit could be allowed or disallowed. Config "node_slots" need to be removed after that.

[sparshev]

It think it will be useful to have scripts executed directly in the cluster nodes, but the lang need to be chosen properly. It's related to the ticket #17 :

• https://github.com/d5/tengo - altered and simplified syntax maybe more suitable to the limited scripts?
• https://github.com/traefik/yaegi - full-featured go lang as a script.
• https://github.com/cosmos72/gomacro - full-featured go lang as a script.
• https://github.com/go-python/gpython - pure go python interpreter, don't like the idea of having different lang that is who knows how supported, but could worth a try.

Overall the choose is not that hard - for example we can get just a simple script lang and after that add support for the other langs as needed.

[sparshev]

Compared the full go script systems as gomacro and yaegi with native and executable go:

• Performance on prime numbers (1000 iterations for primeNumbers(1000)):
  • Compiled (time ./prime):

    real    0m0.082s
    user    0m0.079s
    sys     0m0.008s
    

  • time go run ./prime.go

    real    0m0.148s
    user    0m0.246s
    sys     0m0.138s
    

  • time gomacro ./prime_mod.go (with added main() in the end)

    real    0m2.554s
    user    0m3.110s
    sys     0m0.105s
    

  • time yaegi ./prime.go

    real    0m6.511s
    user    0m7.300s
    sys     0m0.164s
    

• Size of embedded application:
  • gomacro: 30MB
  • yaegi: 17MB
• Found differences between gomacro and yaegi:
  • gomacro don't execute main function of the script by default, while yaegi does

[sparshev]

Right now we defining the hard limit of the node slots in the configuration (NodeSlots), but cloud drivers actually doesn't use the current node resources (cpu/mem/disk). So maybe it will be a good addition to receive the available resources right from the driver and not from the node itself. That will help the rules to get available resources data depends on the label.