fix: testproject runtimetests fails on console platforms v1.1.0 (#1664)

* Squashed commit of the following:

commit 455a756
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Thu Feb 3 12:19:02 2022 -0600

    Revert "test"

    This reverts commit a2f6fca.

commit 1c2d493
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Thu Feb 3 12:17:02 2022 -0600

    style

    white space update

commit ce84aba
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Thu Feb 3 11:34:22 2022 -0600

    test update

    This test was failing on some of the console platforms.  Made adjustments to avoid timing issues.

commit 8ad9b42
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Thu Feb 3 07:53:49 2022 -0600

    fix

    This is a work around for the MTT-1703 issue with RPCs.

commit 85fded2
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Thu Feb 3 05:40:31 2022 -0600

    style

    white spaces.

commit 82606ea
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 22:31:35 2022 -0600

    test update

    Still trying to get rid of IL2CPP error

commit 5915c3e
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 20:13:45 2022 -0600

    test fix

    Trying to figure out weird ILCPP serialization issue.

commit 091d82f
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 18:01:13 2022 -0600

    test refactor

    Needed some MULTIPLAYER_TOOLS helpers that I accidentally excluded.

commit a2f6fca
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 12:03:10 2022 -0600

    test

    disabling all tests in Unity.Netcode.RuntimeTest

commit 5319473
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 12:02:28 2022 -0600

    revert

    reverting as this was a dumb approach.

commit 312a5f5
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 11:19:59 2022 -0600

    test

    Seeing if this fixes the weird issue with the standards failing.

commit 76af4d4
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 09:08:26 2022 -0600

    test

    turning off the Unity.Netcode.RuntimeTests on console platforms temporarily in order to run just the TestProject.RuntimeTests on consoles.

commit 67566d6
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Wed Feb 2 09:06:27 2022 -0600

    refactor

    Removing assembly references not being used.

commit 2480daf
Merge: bee18d3 096f614
Author: Noel Stephens <noel.stephens@unity3d.com>
Date:   Tue Feb 1 20:36:59 2022 -0600

    Merge branch 'develop' into fix/test-roject-runtimetests-standalone-testrunner-build

commit bee18d3
Merge: 2fa12fa e07450f
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 19:28:11 2022 -0600

    Merge branch 'develop' into fix/test-roject-runtimetests-standalone-testrunner-build

commit 2fa12fa
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 19:05:18 2022 -0600

    fix

    This fixes the issues with running the NetworkVariableInitializationOnNetworkSpawnTest when it was run in a stand alone test runner build which should be console friendly at this point.

commit c47ea65
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 19:04:03 2022 -0600

    fix

    Fixing issues with the NetworkPrefabPoolAdditive class that is used in several tests including the NetworkSceneManagerTests.  The issues fixed were related to object references not being around (i.e. NetworkManager) as well as came up with a better way to detect if the coroutine was running in order to determine if it needed to be stopped.

commit 1ee1297
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 17:56:57 2022 -0600

    fix

    The following updates to NetworkSceneManagerDDOLTests in NetworkSceneManagerTests.cs get it passing in a stand alone test runner build which should make it console friendly.

commit 0b955fe
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 17:44:06 2022 -0600

    fix

    This fixes the issues with running NestedNetworkManagerTests in a stand alone test runner build and should run on consoles.

commit 2f18881
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 17:30:41 2022 -0600

    fix

    The following updates to the MessageOrdering tests should get them working on consoles.

commit a967264
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 17:04:25 2022 -0600

    update

    Modifying the TestProject.RuntimeTests assembly to be a runtime only assembly.
    Had to migrate the TestProject.RuntimeTests out of the #if UNITY_EDITOR region in order to be able to compile a stand alone test runner build.

commit d8e5b1f
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 12:22:06 2022 -0600

    refactor

    This commit changes the NetworkObjectParentingTests and ReparentingCubeNetBhv namespace to TestProject.RuntimeTests and uses the standard SceneManager to load the scene as opposed to the editor version.

commit d5efbe6
Author: NoelStephensUnity <73188597+NoelStephensUnity@users.noreply.github.com>
Date:   Tue Feb 1 12:14:48 2022 -0600

    refactor

    This commit migrates the NetworkPrefabGlobalObjectIdHashTests and NetworkObjectGlobalObjectIdHashTests into the TestProject.EditorTests assembly since they depend upon editor specific library calls and could be run as EditMode tests.

* fix

NetworkVariableInitOnNetworkSpawn needed to be updated.

* fix

disabling tests that were specific to snapshot.

* style

Added note about the SIPTransport's DontShowInTransportDropdown

* Update

Found a different way to remove the DontShowInTransportDropdownAttribute which helps to remove the reference to the editor assembly.

Author: NoelStephensUnity

com.unity.netcode.gameobjects/Editor/DontShowInTransportDropdownAttribute.cs

@@ -57,7 +57,7 @@ private void ReloadTransports()
 
                 foreach (var type in types)
                 {
-                    if (type.IsSubclassOf(typeof(NetworkTransport)) && type.GetCustomAttributes(typeof(DontShowInTransportDropdownAttribute), true).Length == 0)
+                    if (type.IsSubclassOf(typeof(NetworkTransport)) && !type.IsSubclassOf(typeof(TestingNetworkTransport)) && type != typeof(TestingNetworkTransport))
                     {
                         m_TransportTypes.Add(type);
                     }

com.unity.netcode.gameobjects/Editor/DontShowInTransportDropdownAttribute.cs.meta

@@ -5,8 +5,8 @@
 [assembly: InternalsVisibleTo("Unity.Netcode.Editor.CodeGen")]
 [assembly: InternalsVisibleTo("Unity.Netcode.Editor")]
 [assembly: InternalsVisibleTo("TestProject.EditorTests")]
-[assembly: InternalsVisibleTo("TestProject.RuntimeTests")]
 [assembly: InternalsVisibleTo("TestProject.ToolsIntegration.RuntimeTests")]
 #endif
+[assembly: InternalsVisibleTo("TestProject.RuntimeTests")]
 [assembly: InternalsVisibleTo("Unity.Netcode.RuntimeTests")]
 

com.unity.netcode.gameobjects/Editor/NetworkManagerEditor.cs

@@ -98,4 +98,11 @@ protected void InvokeOnTransportEvent(NetworkEvent eventType, ulong clientId, Ar
         /// /// <param name="networkManager">optionally pass in NetworkManager</param>
         public abstract void Initialize(NetworkManager networkManager = null);
     }
+
+#if UNITY_INCLUDE_TESTS
+    public abstract class TestingNetworkTransport : NetworkTransport
+    {
+
+    }
+#endif
 }

com.unity.netcode.gameobjects/Runtime/AssemblyInfo.cs

@@ -1,11 +1,9 @@
 using System;
-using Unity.Netcode.Editor;
+
 
 namespace Unity.Netcode.EditorTests
 {
-    // Should probably have one of these for more files? In the future we could use the SIPTransport?
-    [DontShowInTransportDropdown]
-    internal class DummyTransport : NetworkTransport
+    internal class DummyTransport : TestingNetworkTransport
     {
         public override ulong ServerClientId { get; } = 0;
         public override void Send(ulong clientId, ArraySegment<byte> payload, NetworkDelivery networkDelivery)

com.unity.netcode.gameobjects/Runtime/Transports/NetworkTransport.cs

@@ -1,8 +1,5 @@
 using System;
 using System.Collections.Generic;
-#if UNITY_EDITOR
-using Unity.Netcode.Editor;
-#endif
 using UnityEngine;
 
 namespace Unity.Netcode.RuntimeTests
@@ -13,10 +10,7 @@ namespace Unity.Netcode.RuntimeTests
     /// it's designed for the netcode in a way where no networking stack has to be available
     /// it's designed for testing purposes and it's not designed with speed in mind
     /// </summary>
-#if UNITY_EDITOR
-    [DontShowInTransportDropdown]
-#endif
-    public class SIPTransport : NetworkTransport
+    public class SIPTransport : TestingNetworkTransport
     {
         private struct Event
         {

com.unity.netcode.gameobjects/Tests/Editor/NetworkManagerMessageHandlerTests.cs

@@ -3,7 +3,6 @@
     "rootNamespace": "Unity.Netcode.RuntimeTests",
     "references": [
         "Unity.Netcode.Runtime",
-        "Unity.Netcode.Editor",
         "Unity.Netcode.Components",
         "Unity.Collections",
         "UnityEngine.TestRunner",

com.unity.netcode.gameobjects/Tests/Runtime/Transport/SIPTransport.cs

@@ -4,41 +4,32 @@
 using UnityEditor.SceneManagement;
 using UnityEngine.SceneManagement;
 using UnityEngine.TestTools;
+using Unity.Netcode;
 
-namespace Unity.Netcode.RuntimeTests
+namespace TestProject.EditorTests
 {
     public class NetworkObjectGlobalObjectIdHashTests
     {
         private Scene m_TestScene;
 
-        private void OnSceneLoaded(Scene scene, LoadSceneMode mode)
-        {
-            if (scene.name == nameof(NetworkObjectGlobalObjectIdHashTests))
-            {
-                m_TestScene = scene;
-            }
-        }
-
         [UnitySetUp]
         public IEnumerator Setup()
         {
-            SceneManager.sceneLoaded += OnSceneLoaded;
-
-            const string scenePath = "Assets/Tests/Runtime/GlobalObjectIdHash/" + nameof(NetworkObjectGlobalObjectIdHashTests) + ".unity";
+            const string scenePath = "Assets/Tests/Editor/GlobalObjectIdHash/" + nameof(NetworkObjectGlobalObjectIdHashTests) + ".unity";
 
-            yield return EditorSceneManager.LoadSceneAsyncInPlayMode(scenePath, new LoadSceneParameters(LoadSceneMode.Additive));
+            m_TestScene = EditorSceneManager.OpenScene(scenePath, OpenSceneMode.Additive);
             Assert.That(m_TestScene.isLoaded, Is.True);
+            yield return null;
         }
 
         [UnityTearDown]
         public IEnumerator Teardown()
         {
-            SceneManager.sceneLoaded -= OnSceneLoaded;
-
             if (m_TestScene.isLoaded)
             {
-                yield return SceneManager.UnloadSceneAsync(m_TestScene);
+                Assert.True(EditorSceneManager.CloseScene(m_TestScene, true));
             }
+            yield return null;
         }
 
         [Test]

com.unity.netcode.gameobjects/Tests/Runtime/com.unity.netcode.runtimetests.asmdef

@@ -1,50 +1,43 @@
 using System.Collections;
 using System.Collections.Generic;
 using NUnit.Framework;
+using UnityEngine;
 using UnityEditor.SceneManagement;
 using UnityEngine.SceneManagement;
 using UnityEngine.TestTools;
+using Unity.Netcode;
 
-namespace Unity.Netcode.RuntimeTests
+namespace TestProject.EditorTests
 {
     public class NetworkPrefabGlobalObjectIdHashTests
     {
         private Scene m_TestScene;
 
-        private void OnSceneLoaded(Scene scene, LoadSceneMode mode)
-        {
-            if (scene.name == nameof(NetworkPrefabGlobalObjectIdHashTests))
-            {
-                m_TestScene = scene;
-            }
-        }
-
         [UnitySetUp]
         public IEnumerator Setup()
         {
-            SceneManager.sceneLoaded += OnSceneLoaded;
-
-            const string scenePath = "Assets/Tests/Runtime/GlobalObjectIdHash/" + nameof(NetworkPrefabGlobalObjectIdHashTests) + ".unity";
-            yield return EditorSceneManager.LoadSceneAsyncInPlayMode(scenePath, new LoadSceneParameters(LoadSceneMode.Additive));
+            const string scenePath = "Assets/Tests/Editor/GlobalObjectIdHash/" + nameof(NetworkPrefabGlobalObjectIdHashTests) + ".unity";
+            m_TestScene = EditorSceneManager.OpenScene(scenePath, OpenSceneMode.Additive);
+            Assert.That(m_TestScene.isLoaded, Is.True);
+            yield return null;
         }
 
         [UnityTearDown]
         public IEnumerator Teardown()
         {
-            SceneManager.sceneLoaded -= OnSceneLoaded;
-
             if (m_TestScene.isLoaded)
             {
-                yield return SceneManager.UnloadSceneAsync(m_TestScene);
+                Assert.True(EditorSceneManager.CloseScene(m_TestScene, true));
             }
+            yield return null;
         }
 
         [Test]
         public void VerifyUniquenessOfNetworkPrefabs()
         {
             Assert.That(m_TestScene.isLoaded, Is.True);
-
-            var networkManager = NetworkManager.Singleton;
+            var networkManagerObject = GameObject.Find("[NetworkManager]");
+            var networkManager = networkManagerObject.GetComponent<NetworkManager>();
             Assert.That(networkManager, Is.Not.Null);
             Assert.That(networkManager.NetworkConfig, Is.Not.Null);
             Assert.That(networkManager.NetworkConfig.NetworkPrefabs, Is.Not.Null);

testproject/Assets/Tests/Runtime/GlobalObjectIdHash.meta renamed to testproject/Assets/Tests/Editor/GlobalObjectIdHash.meta

@@ -67,8 +67,7 @@ private void RegisterCustomPrefabHandler()
 
         private void DeregisterCustomPrefabHandler()
         {
-            // Register the custom spawn handler?
-            if (EnableHandler && NetworkManager && NetworkManager.PrefabHandler != null)
+            if (EnableHandler && NetworkObject != null && (IsSpawned || (NetworkManager != null && NetworkManager.PrefabHandler != null)))
             {
                 NetworkManager.PrefabHandler.RemoveHandler(ServerObjectToPool);
                 if (IsClient && m_ObjectToSpawn != null)
@@ -84,29 +83,17 @@ private void DeregisterCustomPrefabHandler()
         /// </summary>
         public override void OnDestroy()
         {
-            if (IsServer)
-            {
-                StopCoroutine(SpawnObjects());
-            }
+            InternalStopCoroutine();
             DeregisterCustomPrefabHandler();
 
-
-
-            if (NetworkManager != null && NetworkManager.SceneManager != null)
+            if (NetworkObject != null && NetworkManager != null && NetworkManager.SceneManager != null)
             {
                 NetworkManager.SceneManager.OnSceneEvent -= OnSceneEvent;
             }
 
             base.OnDestroy();
         }
 
-        // Start is called before the first frame update
-        private void Start()
-        {
-            NetworkManager.SceneManager.OnSceneEvent += OnSceneEvent;
-        }
-
-
         /// <summary>
         /// For additive scenes, we only clear out our pooled NetworkObjects if we are migrating them from the ActiveScene
         /// to the scene where this NetworkPrefabPoolAdditive component is instantiated.
@@ -203,10 +190,7 @@ private void CleanNetworkObjects()
         /// </summary>
         private void OnUnloadScene()
         {
-            if (IsServer)
-            {
-                StopCoroutine(SpawnObjects());
-            }
+            InternalStopCoroutine();
 
             // De-register the custom prefab handler
             DeregisterCustomPrefabHandler();
@@ -221,6 +205,7 @@ private void OnUnloadScene()
         /// </summary>
         public override void OnNetworkSpawn()
         {
+            NetworkManager.SceneManager.OnSceneEvent += OnSceneEvent;
             InitializeObjectPool();
             if (IsServer)
             {
@@ -361,15 +346,26 @@ public void UpdateSpawnsPerSecond()
             if (SpawnsPerSecond == 0 && m_IsSpawningObjects)
             {
                 m_IsSpawningObjects = false;
-                StopCoroutine(SpawnObjects());
+                InternalStopCoroutine();
             }
 
         }
 
 
         private void OnDisable()
         {
-            StopCoroutine(SpawnObjects());
+            InternalStopCoroutine();
+        }
+
+        private bool m_CoroutineHasStarted = false;
+
+        private void InternalStopCoroutine()
+        {
+            if (m_CoroutineHasStarted)
+            {
+                StopCoroutine(SpawnObjects());
+                m_CoroutineHasStarted = false;
+            }
         }
 
         /// <summary>
@@ -388,6 +384,7 @@ private IEnumerator SpawnObjects()
             {
                 yield return new WaitForSeconds(m_DelaySpawning - Time.realtimeSinceStartup);
             }
+            m_CoroutineHasStarted = true;
 
             m_IsSpawningObjects = true;
             while (m_IsSpawningObjects)

testproject/Assets/Tests/Runtime/GlobalObjectIdHash/NetworkObjectGlobalObjectIdHashTests.cs renamed to testproject/Assets/Tests/Editor/GlobalObjectIdHash/NetworkObjectGlobalObjectIdHashTests.cs

@@ -4,7 +4,15 @@
     "references": [
         "TestProject",
         "Unity.Netcode.Runtime",
-        "Unity.Netcode.Editor",
         "Unity.Netcode.Components"
-    ]
-}
+    ],
+    "includePlatforms": [],
+    "excludePlatforms": [],
+    "allowUnsafeCode": false,
+    "overrideReferences": false,
+    "precompiledReferences": [],
+    "autoReferenced": true,
+    "defineConstraints": [],
+    "versionDefines": [],
+    "noEngineReferences": false
+}

testproject/Assets/Tests/Runtime/GlobalObjectIdHash/NetworkObjectGlobalObjectIdHashTests.cs.meta renamed to testproject/Assets/Tests/Editor/GlobalObjectIdHash/NetworkObjectGlobalObjectIdHashTests.cs.meta

@@ -95,9 +95,8 @@ public IEnumerator ValidateNetworkObjectSynchronization()
             instanceNetworkObject.NetworkManagerOwner = m_ServerNetworkManager;
             instanceNetworkObject.Spawn();
             var serverobjectToNotDestroyBehaviour = objectInstance.GetComponent<ObjectToNotDestroyBehaviour>();
-
-            int nextFrameNumber = Time.frameCount + 32;
-            yield return new WaitUntil(() => Time.frameCount >= nextFrameNumber);
+            var waitForTick = new WaitForSeconds(1.0f / m_ServerNetworkManager.NetworkConfig.TickRate);
+            yield return waitForTick;
 
             Assert.IsTrue(objectInstance.scene.name == "DontDestroyOnLoad");
 
@@ -106,24 +105,45 @@ public IEnumerator ValidateNetworkObjectSynchronization()
                 networkManager.StartClient();
             }
 
-            yield return MultiInstanceHelpers.Run(MultiInstanceHelpers.WaitForClientsConnected(m_ClientNetworkManagers));
+            yield return MultiInstanceHelpers.WaitForClientsConnected(m_ClientNetworkManagers);
 
-            nextFrameNumber = Time.frameCount + 32;
-            yield return new WaitUntil(() => Time.frameCount >= nextFrameNumber);
-
-            foreach (var networkManager in m_ClientNetworkManagers)
+            yield return waitForTick;
+            var timeOut = Time.realtimeSinceStartup + 2.0f;
+            var timedOut = false;
+            while (!timedOut)
             {
-                foreach (var spawnedObject in networkManager.SpawnManager.SpawnedObjectsList)
+                var allClientConditionsHaveBeenReached = true;
+                foreach (var networkManager in m_ClientNetworkManagers)
                 {
-                    if (spawnedObject.NetworkManager == networkManager && spawnedObject.gameObject.name.Contains("DontDestroyOnLoadObject"))
+                    foreach (var spawnedObject in networkManager.SpawnManager.SpawnedObjectsList)
                     {
-                        Assert.IsTrue(spawnedObject.gameObject.scene.name == "DontDestroyOnLoad");
-                        var objectToNotDestroyBehaviour = spawnedObject.gameObject.GetComponent<ObjectToNotDestroyBehaviour>();
-                        Assert.Greater(objectToNotDestroyBehaviour.CurrentPing, 0);
-                        Assert.AreEqual(serverobjectToNotDestroyBehaviour.CurrentPing, objectToNotDestroyBehaviour.CurrentPing);
+                        if (spawnedObject.NetworkManager == networkManager && spawnedObject.gameObject.name.Contains("DontDestroyOnLoadObject"))
+                        {
+                            if (spawnedObject.gameObject.scene.name != "DontDestroyOnLoad")
+                            {
+                                allClientConditionsHaveBeenReached = false;
+                                break;
+                            }
+                            var objectToNotDestroyBehaviour = spawnedObject.gameObject.GetComponent<ObjectToNotDestroyBehaviour>();
+                            if (objectToNotDestroyBehaviour.CurrentPing == 0 || serverobjectToNotDestroyBehaviour.CurrentPing != objectToNotDestroyBehaviour.CurrentPing)
+                            {
+                                allClientConditionsHaveBeenReached = false;
+                                break;
+                            }
+                        }
                     }
                 }
+
+                if (allClientConditionsHaveBeenReached)
+                {
+                    break;
+                }
+
+                yield return waitForTick;
+
+                timedOut = timeOut < Time.realtimeSinceStartup;
             }
+            Assert.False(timedOut, "Timed out while waiting for all client conditions to be reached!");
         }
     }
 }