VNF resource consumption

[qarawlus]

Right now the node capacities are fixed number. Once a flow arrives at the node, the flow's data rate is deducted from the node's capacity regardless of the SF that it is being processed at.

Since this model is not realistic, the next step is to add non-fixed capacity handling. In this non-fixed context, the deduction from the node's capacity should be done via a function based on several parameters and should also be per SF.

My initial thoughts regarding this are as follows:

• Nodes have fixed capacity numbers (e.g. simulating real fixed CPU capacity or Memory capacity).
• Each SF has a defined idle resource consumption figure to simulate that placed SFs actually consume resources even without processing flows.
• Once a flow arrives to an SF, it's data rate is passed to a function defined within the SF definition, and the resulting resource consumption figure is then deducted from the node's capacity.
• Another idea is to also define the SF processing delays in relation to the node's capacity. This means that higher capacity nodes have lower processing delays for the SFs than nodes with lower capacity.
• Another idea is to specify maximum capacity cap for each SF so that it does not consume more than it is allowed to.

[qarawlus]

@stefanbschneider Let me know what you think

[stefanbschneider]

Looks good! A few minor comments:

Nodes have fixed capacity numbers (e.g. simulating real fixed CPU capacity or Memory capacity).

Let's focus on a single, abstract node resource. Should be quite trivial to extend to more different resource kinds.

Each SF has a defined idle resource consumption figure to simulate that placed SFs actually consume resources even without processing flows.

Yes, but idle consumption may be (close to) zero. This should be returned from the function specifying the VNF resource consumption based on the data rate to handle.

Probably it's a good idea to start with the first 3 bullets (not now), and postpone the latter 2. Impact on processing delay.

[stefanbschneider]

Idea for specifying generic resource consumption functions (for later):

• Implement function for each SF in a separate Python module and function with a fixed function signature, eg, def sf_resource_req(load).
• This allows implementing any kind of generic function: Linear, idle, anything with ML models, ...
• In the service spec yaml, for each SF there needs to be a path to the corresponding Python module implementation
• There should be default behavior that allows to omit implementing and referencing a Python module and function. Eg, simple static resources?

[stefanbschneider]

@ldklenner @qarawlus Let's use this issue to agree on how to implement individual VNF resource consumptions. Once this is clear, we can still see who needs it first/who'll implement it.

[stefanbschneider]

My approach would be, as sketched above, to extend the yaml file defining the service and SFs to optionally include a path to a Python module that defines a resource consumption function. Eg,

...

sf_list:
  a:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
    resource_module: 'resource_consumption_a.py'
  b:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
    resource_module: 'resource_consumption_b.py'
  c:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0

Then we could use importlib do load the Python module dynamically: https://docs.python.org/3/library/importlib.html#importlib.import_module or https://stackoverflow.com/a/46483074/2745116

In the python module, there should be a function def sf_resource_req(load) that returns the required resources depending on the data rate/load. If no such function is defined in the yaml, we should use a simple linear model as default.

The returned resource requirements would then be used to check against the remaining node resources instead of the flow data rate: https://github.com/RealVNF/coordination-simulation/blob/master/src/coordsim/simulation/flowsimulator.py#L196

For now, we should start with a single node resource.

[stefanbschneider]

An alternative, which is probably simpler than individual Python modules per SF, we could also define a single Python module for all SFs like so:

sf_list:
  a:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
  b:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
  c:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
sf_resource_req_module: "sfc1_resource_req.py"

Then inside the module, the function would need to return the required resources based on the load and the specified SF name: def sf_resource_req(sf, load).
It should still work without defining any Python module.

This would be simpler as we only need to write, specify, and import a single Python module.

What do you think?

[qarawlus]

Why a separate module file? What would be there in that function that would require a separate module? Why can't we parameterize it so that we only have one sf_resource_req(sf, params) function and pass parameters from the yaml file and process it there. What would a possible scenario be where the SF resource requirements would be so complex and in such different ways that would create a need for separate modules to process them?

[stefanbschneider]

In an ongoing thesis, we're using trained ML models that were trained on benchmarking data to predict more resource requirements. It would also allow us to specify non-linear, eg, logarithmic resource consumption.
Here the separate Python module would offer the largest flexibility.

But I agree in 90% of the cases, a linear function would be enough. That's why I'd use this as default.

[ldklenner]

I think that different modules would provide a clean and flexible way of function definitions. The importlib should make loading these files managable.
The reader, when constructing the sf_list, would try to load the specified files, access the resource function and store a reference in a new entry, similar to:

sf_list[sf_name]['resource_function'] = resource_function

If there is no file or function, a default function is used.
The loading of files would require to add a path parameter to the simulator, describing where the resource function files are located.

A more intricate problem presents the calculation of used capacity. As every sf might have a different resource function a node has to record the current load of every placed sf. When checking if a flow can be processed, the individual sf loads have to be applied to their associcated resource functions.

[stefanbschneider]

Yes, storing the function references like this would make using them later quite convenient.

True, calculation is indeed a bit more complicated. But I guess you approach would be the best solution: Keep track of load per SF per node by updating it whenever a flow starts or stops processing. Whenever a new flow arrives for processing, calculate the current resource consumption based on the increased load per SF if the flow would be added and the SF resource functions to check if the overall resource consumption would be below the capacity.

[stefanbschneider]

Closed by #78