Added VNF resource consumption functions

[ldklenner]

VNF/SF can no specifiy a resource function which defines the needed capacity dependent on the individual SF load. If no resource function is specified, the default function f(x)=x will be used. When using resource functions a path parameter must be passed to the simulator.init function.

 a:
    processing_delay_mean: 5.0
    processing_delay_stdev: 0.0
    resource_function_id: A

The reader when constructing the sf_list will try to load "A.py" in the specified path.
"A.py" has to contain a resource_function(load) function.

Note: do not include ".py" in the yaml resource_function_id field.
Note: resource functions are optional. Passing no path to the simulator will assign the default function to all SFs with warnings.

[ldklenner]

Local tests work and produced the intended output. Travis tests encounter severe problems, do to the fact that "test_simulator.py" do not use the simulator interface function simulator.apply.

simulator.apply is used to construct the list of available SF at each node from the placement parameter. Without the function call the list stays empty. The flowsimulator when attempting to record the current load of a SF will fail as the SF is placed but not listed as available.

[ldklenner]

SF load is now recorded from the beginning. Next commit will change travis.yaml to explicit load resource funtions.

[stefanbschneider]

Nice work! Is this ready to merge?

@qarawlus Please have a look at the changes, so you are aware 👍

[ldklenner]

Now it should be ready to merge.

[stefanbschneider]

Closes #54

[stefanbschneider]

What does this do exactly?

[ldklenner]

Each node stores a list of SFs that where placed and there current load as a node attribute. When changing the placement the list has to be updated.

[stefanbschneider]

So the .get(sf, {'load': 0.0}) gets the current load of the SF at the node and defaults to 0 if the SF wasn't available?

[stefanbschneider]

When testing the changes with our RL algorithm, I ran into the following error:

File "c:\users\stefan\git-repos\work\swc\coordination-simulation\src\coordsim\simulation\flowsimulator.py", line 241, in process_flow
    self.params.network.nodes[current_node_id]['available_sf'][sf]['load'] -= flow.dr
KeyError: 'c'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "c:\users\stefan\git-repos\work\swc\coordination-simulation\src\coordsim\simulation\flowsimulator.py", line 120, in pass_flow
    yield self.env.process(self.process_flow(flow, sfc))
KeyError: 'c'

Seems like there's something wrong with the available SF at each node. I'm going through the code to better understand what's the issue right now. Thankful for any tips you might have.

[stefanbschneider]

I think I have the error:

When reducing the load at an SF because a flow is done processing, self.params.network.nodes[current_node_id]['available_sf'][sf]['load'] -= flow.dr
is called: https://github.com/RealVNF/coordination-simulation/blob/master/src/coordsim/simulation/flowsimulator.py#L241

It's possible that the placement changes in between starting and stopping to process a flow and that at the time of accessing the dict, the SF is no longer available at the node.

[ldklenner]

It's possible that the placement changes in between starting and stopping to process a flow and that at the time of accessing the dict, the SF is no longer available at the node.

Yes, i think that is exactly what happens. In the previous version a SF could be removed while a flow is still in process without violating capacity demands, the capacity when finished processing would simply increased. With this version the load has to be recorded precisely.

[stefanbschneider]

Yep. I'm not sure what would be the best way to handle this.

• a) Either, we're strict and drop any flows that are being processed when a VNF is moved
• b) Or we stick to the previous model and allow flows to finish processing

[stefanbschneider]

a) Drop flows when VNF is removed

• Would be easy to implement
• But would basically always lead to dropped flows when changing the placement
• Not necessarily realistic

b) Allow flows to finish processing

• Could argue that it's more realistic to finish VNF "gracefully", ie, don't send any new flows but allow active flows to finish before stopping VNF
• But then we should still account for the resource consumption of the VNF while it's still processing
• Not sure how to do this without too much effort. Currently we only keep track of available SFs, which makes sense.
• Of course, we could also simply ignore the resource consumption of the SF in this case, since now new flows will be scheduled to it anyway. But not 100% accurate

Any thoughts or ideas?

[stefanbschneider]

I need to get rid of the error, so my current fix goes with b) and allows flows to finish processing when an SF is removed. #80

But it doesn't account for resource consumption of these (almost) removed VNFs and simply ignores it. Not perfect, but hopefully not a big issue - should keep it in mind anyways. This is in line with what we had previously.

Any better solution is more than welcome

[ldklenner]

I think option a) would lead to more serious consequences especially when a flow is simultaneously processed by multiple VNFs.

Option b) is the way to go. The primary decision if a flow gets processed is based on the placement. If a SF is no longer included in the placement a flow will not be processed by it.
What we could do is: keep processing SFs in the available SF list even if there no longer included in the placement. When the load is decreased to 0 and the SF is no longer in the placement, the SF will be removed from the available list.

[stefanbschneider]

What we could do is: keep processing SFs in the available SF list even if there no longer included in the placement. When the load is decreased to 0 and the SF is no longer in the placement, the SF will be removed from the available list.

Yes, that's also what I thought. Not sure how much effort this would be?

[ldklenner]

I'm looking into it.