Add initial solution with ttc values as attacker penalties

[mrkickling]

• Implement attacker ttc penalty
• Write tests

if setting ttc_values_as_attacker_penalty is enabled,
attackers get rewards as before but additionally gets negative rewards for attack step TTCs.

Attack steps are successfully executed instantly even though they have a TTC value set, but the TTC value is given as a penalty to the attacker agent.

Requires TTCMode PRE_SAMPLE or EXPECTED_VALUE so there are ttc values pre calculated.

[sandorstormen]

Can I set ttc_values_as_attacker_penalty to True and still select how TTC values are sampled?

[mrkickling]

Can I set ttc_values_as_attacker_penalty to True and still select how TTC values are sampled?

Yes, but the current implementation on this branch requires you to pick either PRE_SAMPLE or EXPECTED_VALUE, since there otherwise isn't any pre-calculated ttc value. I explained a bit more detail in the top.

Note: this is not necessarily the final solution, we could chose the other approach that just penalizes attackers with 1 point for each step they attempt.

[sandorstormen]

this is not necessarily the final solution, we could chose the other approach that just penalizes attackers with 1 point for each step they attempt.

But isn't this solution already implemented? You won't be able to train an attacker RL agent with ttc_values_as_attacker_penalty=False otherwise.

[mrkickling]

No, an attacker agent is not penalized with 1 point per step it takes, and never was. It is not penalized at all as it is now.