Low level memory allocation every frame

[szantner]

Describe the bug
I detect seemingly low level memory allocations in the EnvironmentStep call every frame as seen in the screenshot using Deep Profile in Profiler

To Reproduce
Using multiple agents in inference mode.

Console logs / stack traces

Screenshots

Environment (please complete the following information):

• Unity Version: Unity 2020.2.2f1
• OS + version: Windows 10
• ML-Agents version: 1.0.6

[chriselion]

Hi,
I think you're right that we're allocating a BinaryWriter and MemoryStream on each step, even if there are no messages to process:

ml-agents/com.unity.ml-agents/Runtime/SideChannels/SideChannelManager.cs

Lines 121 to 126 in 30bcb01

	internal static byte[] GetSideChannelMessage(Dictionary<Guid, SideChannel> sideChannels)
	{
	using (var memStream = new MemoryStream())
	{
	using (var binaryWriter = new BinaryWriter(memStream))
	{

We can definitely early out in that case (or maybe reuse the objects). Is that the main allocation you're seeing?

[chriselion]

We've got this logged in our internal tracker as MLA-1720. I'll probably get to it this week, but it should be pretty straightforward if you want to submit a PR for it instead.

[szantner]

Hi Chris,

Thank you for the quick response. It is probably best if I leave the modifications for you. :)

Btw - and maybe that should be a separate issue - but I saw a strange memory allocation too in another part of ml_agents, but that I managed to fix on my side. So if I did the following, that caused memory allocations:

sensor.AddObservation(_observationLadarInfoOffRoad);

where private List<float> _observationLadarInfoOffRoad;

When I added the observations in a for loop myself, the memory allocation disappeared:

for (int i = 0, length = _observationLadarInfoOffRoad.Count; i < length; i++) { sensor.AddObservation(_observationLadarInfoOffRoad[i]); }

[chriselion]

I think I see what you're saying. This is happening in your Agent'sCollectObservations(VectorSensor sensor) method, right? If you use sensor.AddObservation(_observationLadarInfoOffRoad); it would treat the List as an IEnumerable:

ml-agents/com.unity.ml-agents/Runtime/Sensors/VectorSensor.cs

Lines 167 to 173 in 30bcb01

	public void AddObservation(IEnumerable<float> observation)
	{
	foreach (var f in observation)
	{
	AddFloatObs(f);
	}
	}

which might allocate memory in the foreach loop; I need to look into it further. Your approach is equivalent but definitely doesn't need to allocate anything.

[chriselion]

Thanks, I was able to confirm that by modifying one of our examples:

I think we can add an AddObservation(IList<float> observation) overload to VectorSensor, which should help get rid of this allocation.

[szantner]

Hi Chris, great to hear that it wasn't a false alarm. Thank you for the super quick responses!

[chriselion]

No problem, thanks for pointing me in the right direction. I have MLA-1724 created for this.

[qiwu57kevin]

Just curious, how do you get this profiler windows? That looks pretty helpful in checking each function call.

[chriselion]

https://docs.unity3d.com/Manual/ProfilerWindow.html with Deep Profile enabled.

[chriselion]

The SideChannel allocations are addressed in #4886
I added an overload for VectorSensor.AddObservation in #4887 (and also cleaned up a lot more garbage during inference).