Closed
Description
Describe the bug
I have a script that instantiates a number of environments, agents, and targets. I add the needed scripts like DesicionRequestor, Custom Agent, and BehaviourParameters in code. When the trainer is offline (only inference on previously trained model), this setup works perfectly. When I try this without a model and connect it to the Python trainer. The error ArgumentNullException: Value cannot be null. - Parameter name: values is thrown. When I add the scripts in the inspector, the setup works so I guess my agent class is not faulty.
To Reproduce
Steps to reproduce the behavior:
- Choose an environment
- Remove All TrainingArea's
- Copy C# Code into script and attach to empty GameObject
- Assign correct prefabs to script
- Start Python trainer with standard config
- Start Play Mode
Console logs / stack traces
This error is thrown continuously with only 1 environment.
ArgumentNullException: Value cannot be null.
Parameter name: values
Google.Protobuf.ProtoPreconditions.CheckNotNull[T] (T value, System.String name) (at T:/src/github/protobuf/csharp/src/Google.Protobuf/ProtoPreconditions.cs:55)
Google.Protobuf.Collections.RepeatedField`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] values) (at T:/src/github/protobuf/csharp/src/Google.Protobuf/Collections/RepeatedField.cs:305)
MLAgents.GrpcExtensions.ToProto (MLAgents.Policies.BrainParameters bp, System.String name, System.Boolean isTraining) (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Communicator/GrpcExtensions.cs:79)
MLAgents.RpcCommunicator.GetTempUnityRlInitializationOutput () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Communicator/RpcCommunicator.cs:473)
MLAgents.RpcCommunicator.SendBatchedMessageHelper () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Communicator/RpcCommunicator.cs:321)
MLAgents.RpcCommunicator.DecideBatch () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Communicator/RpcCommunicator.cs:260)
MLAgents.Policies.RemotePolicy.DecideAction () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Policies/RemotePolicy.cs:39)
MLAgents.Agent.DecideAction () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Agent.cs:805)
MLAgents.Academy.EnvironmentStep () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Academy.cs:467)
MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at C:/Users/Joach/Documents/repositories/artifishal_intelligence/com.unity.ml-agents/Runtime/Academy.cs:34)
Environment (please complete the following information):
- OS + version: Windows 10 + Trainer in Docker
- ML-Agents version: ML-Agents v0.15.1
- TensorFlow version: 2.0.1
Trainer config:
default:
trainer: ppo
batch_size: 1024
beta: 5.0e-3
buffer_size: 10240
epsilon: 0.2
hidden_units: 128
lambd: 0.95
learning_rate: 3.0e-4
learning_rate_schedule: linear
max_steps: 20.0e4
memory_size: 256
normalize: false
num_epoch: 3
num_layers: 2
time_horizon: 64
sequence_length: 64
summary_freq: 10000
use_recurrent: false
vis_encode_type: simple
reward_signals:
extrinsic:
strength: 1.0
gamma: 0.99
C#-Code:
using MLAgents.Policies;
using UnityEngine;
using MLAgents;
using Barracuda;
public class Training_RL_IL_CL : MonoBehaviour
{
public int numberOfInstances = 1;
public GameObject environmentPrefab;
public GameObject agentPrefab;
public GameObject goalPrefab;
public NNModel trainedModel;
void Awake()
{
for (int i = 0; i < numberOfInstances; i++)
{
// Instantiating & positioning
var position = new Vector3((i % 10) * 12, (i / 10) * 12, 0);
var env = Instantiate(environmentPrefab, position, Quaternion.identity);
var agent = Instantiate(agentPrefab, Vector3.zero, Quaternion.identity);
// Deactivate agent so training does not start yet
agent.SetActive(false);
var goal = Instantiate(goalPrefab, Vector3.zero, Quaternion.identity);
agent.transform.parent = env.transform;
goal.transform.parent = env.transform;
// Add scripts
agent.AddComponent<FishAgent>();
// agent.AddComponent<BehaviorParameters>(); -> Not needed, added automatically
agent.AddComponent<DecisionRequester>();
// Get scripts to modify
var beh = agent.GetComponent<BehaviorParameters>();
var cus = agent.GetComponent<FishAgent>();
// Setup brain
beh.behaviorName = "RL-Fish";
var brain = new BrainParameters();
brain.vectorActionSpaceType = SpaceType.Continuous;
brain.vectorObservationSize = 9;
brain.vectorActionSize = new[] { 3 };
// Load model if needed
if (trainedModel != null)
beh.model = trainedModel;
// Use for later
beh.inferenceDevice = InferenceDevice.CPU;
beh.TeamId = 0;
beh.useChildSensors = true;
// Setup custom script (Agent)
cus.target = goal.transform;
cus.swimSpeed = 3;
cus.rotationSpeed = 100;
// Set to use new brain
beh.brainParameters = brain;
// Activate the completed Agent
agent.SetActive(true);
}
}
}